Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/with_whim/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/with_whim/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=0),
 'dropout': 0.25,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/train.npz'],
 'features_scaling': True,
 'features_size': None,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'grad_clip': None,
 'hidden_size': 1800,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'metrics': ['prc-auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': False,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/with_whim',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/test.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/val.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.344032
Epoch 1
Validation prc-auc = 0.370081
Epoch 2
Validation prc-auc = 0.449001
Epoch 3
Validation prc-auc = 0.367273
Epoch 4
Loss = 5.5651e-01, PNorm = 87.5388, GNorm = 3.6482, lr_0 = 6.5000e-04
Validation prc-auc = 0.364047
Epoch 5
Validation prc-auc = 0.272900
Epoch 6
Validation prc-auc = 0.374953
Epoch 7
Validation prc-auc = 0.436363
Epoch 8
Validation prc-auc = 0.181939
Epoch 9
Loss = 6.0208e-01, PNorm = 87.8841, GNorm = 3.2176, lr_0 = 9.9743e-04
Validation prc-auc = 0.252503
Epoch 10
Validation prc-auc = 0.451271
Epoch 11
Validation prc-auc = 0.451992
Epoch 12
Validation prc-auc = 0.312510
Epoch 13
Validation prc-auc = 0.201279
Epoch 14
Loss = 5.1260e-01, PNorm = 88.2677, GNorm = 1.6770, lr_0 = 9.8890e-04
Validation prc-auc = 0.393631
Epoch 15
Validation prc-auc = 0.379470
Epoch 16
Validation prc-auc = 0.375507
Epoch 17
Validation prc-auc = 0.365772
Epoch 18
Validation prc-auc = 0.375522
Epoch 19
Loss = 2.6817e-01, PNorm = 88.5630, GNorm = 1.0661, lr_0 = 9.8045e-04
Validation prc-auc = 0.408728
Epoch 20
Validation prc-auc = 0.429089
Epoch 21
Validation prc-auc = 0.405120
Epoch 22
Validation prc-auc = 0.384412
Epoch 23
Validation prc-auc = 0.393887
Epoch 24
Loss = 2.0151e-01, PNorm = 88.7876, GNorm = 1.1419, lr_0 = 9.7207e-04
Validation prc-auc = 0.426517
Epoch 25
Validation prc-auc = 0.427547
Epoch 26
Validation prc-auc = 0.405659
Epoch 27
Validation prc-auc = 0.375623
Epoch 28
Validation prc-auc = 0.404763
Epoch 29
Loss = 2.7280e-01, PNorm = 88.9771, GNorm = 1.9231, lr_0 = 9.6376e-04
Validation prc-auc = 0.370781
Epoch 30
Validation prc-auc = 0.391383
Epoch 31
Validation prc-auc = 0.395607
Epoch 32
Validation prc-auc = 0.412493
Epoch 33
Validation prc-auc = 0.427754
Epoch 34
Loss = 2.1394e-01, PNorm = 89.1781, GNorm = 2.5179, lr_0 = 9.5552e-04
Validation prc-auc = 0.420950
Epoch 35
Validation prc-auc = 0.428178
Epoch 36
Validation prc-auc = 0.368874
Epoch 37
Validation prc-auc = 0.226992
Epoch 38
Validation prc-auc = 0.330959
Epoch 39
Loss = 2.3946e-01, PNorm = 89.4080, GNorm = 1.7784, lr_0 = 9.4735e-04
Validation prc-auc = 0.388333
Epoch 40
Validation prc-auc = 0.385753
Epoch 41
Validation prc-auc = 0.327511
Epoch 42
Validation prc-auc = 0.367560
Epoch 43
Validation prc-auc = 0.419578
Epoch 44
Loss = 2.5178e-01, PNorm = 89.6420, GNorm = 1.9268, lr_0 = 9.3925e-04
Validation prc-auc = 0.427129
Epoch 45
Validation prc-auc = 0.447394
Epoch 46
Validation prc-auc = 0.453394
Epoch 47
Validation prc-auc = 0.457110
Epoch 48
Validation prc-auc = 0.441124
Epoch 49
Loss = 1.4301e-01, PNorm = 89.9085, GNorm = 0.7441, lr_0 = 9.3122e-04
Validation prc-auc = 0.416676
Epoch 50
Validation prc-auc = 0.390695
Epoch 51
Validation prc-auc = 0.319739
Epoch 52
Validation prc-auc = 0.403093
Epoch 53
Validation prc-auc = 0.439282
Epoch 54
Loss = 2.8943e-01, PNorm = 90.1989, GNorm = 2.4909, lr_0 = 9.2326e-04
Validation prc-auc = 0.420451
Epoch 55
Validation prc-auc = 0.403481
Epoch 56
Validation prc-auc = 0.348076
Epoch 57
Validation prc-auc = 0.430942
Epoch 58
Validation prc-auc = 0.419693
Epoch 59
Loss = 1.7478e-01, PNorm = 90.5076, GNorm = 1.8071, lr_0 = 9.1537e-04
Validation prc-auc = 0.423906
Epoch 60
Validation prc-auc = 0.432835
Epoch 61
Validation prc-auc = 0.418011
Epoch 62
Validation prc-auc = 0.415241
Epoch 63
Validation prc-auc = 0.420444
Epoch 64
Loss = 4.4415e-02, PNorm = 90.8035, GNorm = 0.5989, lr_0 = 9.0754e-04
Validation prc-auc = 0.405779
Epoch 65
Validation prc-auc = 0.415617
Epoch 66
Validation prc-auc = 0.400656
Epoch 67
Validation prc-auc = 0.424575
Epoch 68
Validation prc-auc = 0.435093
Epoch 69
Loss = 5.9192e-02, PNorm = 91.0660, GNorm = 1.0938, lr_0 = 8.9978e-04
Validation prc-auc = 0.412089
Epoch 70
Validation prc-auc = 0.395827
Epoch 71
Validation prc-auc = 0.404584
Epoch 72
Validation prc-auc = 0.396997
Epoch 73
Validation prc-auc = 0.402808
Epoch 74
Loss = 8.8626e-02, PNorm = 91.3174, GNorm = 0.5628, lr_0 = 8.9209e-04
Validation prc-auc = 0.411903
Epoch 75
Validation prc-auc = 0.438556
Epoch 76
Validation prc-auc = 0.410783
Epoch 77
Validation prc-auc = 0.414154
Epoch 78
Validation prc-auc = 0.331741
Epoch 79
Loss = 2.3506e-01, PNorm = 91.5766, GNorm = 1.4658, lr_0 = 8.8447e-04
Validation prc-auc = 0.406740
Epoch 80
Validation prc-auc = 0.433251
Epoch 81
Validation prc-auc = 0.450468
Epoch 82
Validation prc-auc = 0.440225
Epoch 83
Validation prc-auc = 0.431489
Epoch 84
Loss = 9.1631e-02, PNorm = 91.8917, GNorm = 2.1223, lr_0 = 8.7691e-04
Validation prc-auc = 0.421758
Epoch 85
Validation prc-auc = 0.406160
Epoch 86
Validation prc-auc = 0.422655
Epoch 87
Validation prc-auc = 0.436023
Epoch 88
Validation prc-auc = 0.429249
Epoch 89
Loss = 5.1516e-02, PNorm = 92.1864, GNorm = 0.9043, lr_0 = 8.6941e-04
Validation prc-auc = 0.437745
Epoch 90
Validation prc-auc = 0.424588
Epoch 91
Validation prc-auc = 0.437606
Epoch 92
Validation prc-auc = 0.406585
Epoch 93
Validation prc-auc = 0.425372
Epoch 94
Loss = 1.0127e-01, PNorm = 92.4613, GNorm = 1.9061, lr_0 = 8.6198e-04
Validation prc-auc = 0.436035
Epoch 95
Validation prc-auc = 0.433944
Epoch 96
Validation prc-auc = 0.418431
Epoch 97
Validation prc-auc = 0.383330
Epoch 98
Validation prc-auc = 0.407262
Epoch 99
Loss = 1.1277e-01, PNorm = 92.7192, GNorm = 1.3225, lr_0 = 8.5461e-04
Validation prc-auc = 0.400893
Epoch 100
Validation prc-auc = 0.404315
Epoch 101
Validation prc-auc = 0.437663
Epoch 102
Validation prc-auc = 0.432243
Epoch 103
Validation prc-auc = 0.435734
Epoch 104
Loss = 4.0195e-02, PNorm = 92.9233, GNorm = 0.7237, lr_0 = 8.4730e-04
Validation prc-auc = 0.427561
Epoch 105
Validation prc-auc = 0.445734
Epoch 106
Validation prc-auc = 0.443809
Epoch 107
Validation prc-auc = 0.443775
Epoch 108
Validation prc-auc = 0.452443
Epoch 109
Loss = 3.6256e-02, PNorm = 93.1085, GNorm = 0.8024, lr_0 = 8.4006e-04
Validation prc-auc = 0.454511
Epoch 110
Validation prc-auc = 0.412824
Epoch 111
Validation prc-auc = 0.417579
Epoch 112
Validation prc-auc = 0.419464
Epoch 113
Validation prc-auc = 0.418261
Epoch 114
Loss = 3.4932e-02, PNorm = 93.2723, GNorm = 0.5842, lr_0 = 8.3288e-04
Validation prc-auc = 0.416707
Epoch 115
Validation prc-auc = 0.414271
Epoch 116
Validation prc-auc = 0.395565
Epoch 117
Validation prc-auc = 0.429931
Epoch 118
Validation prc-auc = 0.430082
Epoch 119
Loss = 2.1415e-02, PNorm = 93.4252, GNorm = 1.0029, lr_0 = 8.2576e-04
Validation prc-auc = 0.442086
Epoch 120
Validation prc-auc = 0.444639
Epoch 121
Validation prc-auc = 0.436829
Epoch 122
Validation prc-auc = 0.453596
Epoch 123
Validation prc-auc = 0.443728
Epoch 124
Loss = 8.1547e-02, PNorm = 93.5622, GNorm = 0.0963, lr_0 = 8.1870e-04
Validation prc-auc = 0.443355
Epoch 125
Validation prc-auc = 0.437258
Epoch 126
Validation prc-auc = 0.382099
Epoch 127
Validation prc-auc = 0.390463
Epoch 128
Validation prc-auc = 0.442962
Epoch 129
Loss = 4.9623e-02, PNorm = 93.7370, GNorm = 1.1687, lr_0 = 8.1170e-04
Validation prc-auc = 0.451872
Epoch 130
Validation prc-auc = 0.461339
Epoch 131
Validation prc-auc = 0.452898
Epoch 132
Validation prc-auc = 0.456928
Epoch 133
Validation prc-auc = 0.450705
Epoch 134
Loss = 1.7385e-02, PNorm = 93.9703, GNorm = 0.9916, lr_0 = 8.0476e-04
Validation prc-auc = 0.438488
Epoch 135
Validation prc-auc = 0.414797
Epoch 136
Validation prc-auc = 0.403442
Epoch 137
Validation prc-auc = 0.419448
Epoch 138
Validation prc-auc = 0.431087
Epoch 139
Loss = 1.2287e-01, PNorm = 94.1683, GNorm = 3.8710, lr_0 = 7.9788e-04
Validation prc-auc = 0.439286
Epoch 140
Validation prc-auc = 0.439165
Epoch 141
Validation prc-auc = 0.441660
Epoch 142
Validation prc-auc = 0.433234
Epoch 143
Validation prc-auc = 0.437990
Epoch 144
Loss = 9.2265e-02, PNorm = 94.4221, GNorm = 1.8033, lr_0 = 7.9106e-04
Validation prc-auc = 0.438044
Epoch 145
Validation prc-auc = 0.441167
Epoch 146
Validation prc-auc = 0.435630
Epoch 147
Validation prc-auc = 0.434728
Epoch 148
Validation prc-auc = 0.422960
Epoch 149
Loss = 1.0687e-01, PNorm = 94.6829, GNorm = 1.7292, lr_0 = 7.8430e-04
Validation prc-auc = 0.410713
Epoch 150
Validation prc-auc = 0.433073
Epoch 151
Validation prc-auc = 0.434542
Epoch 152
Validation prc-auc = 0.413341
Epoch 153
Validation prc-auc = 0.418334
Epoch 154
Loss = 4.5545e-02, PNorm = 94.9116, GNorm = 1.5295, lr_0 = 7.7759e-04
Validation prc-auc = 0.438373
Epoch 155
Validation prc-auc = 0.431072
Epoch 156
Validation prc-auc = 0.391494
Epoch 157
Validation prc-auc = 0.382345
Epoch 158
Validation prc-auc = 0.394471
Epoch 159
Loss = 6.1095e-02, PNorm = 95.1189, GNorm = 0.0595, lr_0 = 7.7095e-04
Validation prc-auc = 0.413912
Epoch 160
Validation prc-auc = 0.424545
Epoch 161
Validation prc-auc = 0.435616
Epoch 162
Validation prc-auc = 0.435453
Epoch 163
Validation prc-auc = 0.453421
Epoch 164
Loss = 3.0891e-02, PNorm = 95.2848, GNorm = 0.9811, lr_0 = 7.6436e-04
Validation prc-auc = 0.464812
Epoch 165
Validation prc-auc = 0.467568
Epoch 166
Validation prc-auc = 0.468404
Epoch 167
Validation prc-auc = 0.456359
Epoch 168
Validation prc-auc = 0.455486
Epoch 169
Loss = 5.2052e-03, PNorm = 95.4044, GNorm = 0.9049, lr_0 = 7.5782e-04
Validation prc-auc = 0.440414
Epoch 170
Validation prc-auc = 0.443504
Epoch 171
Validation prc-auc = 0.446120
Epoch 172
Validation prc-auc = 0.463537
Epoch 173
Validation prc-auc = 0.468414
Epoch 174
Loss = 2.6888e-02, PNorm = 95.4898, GNorm = 0.5745, lr_0 = 7.5134e-04
Validation prc-auc = 0.444700
Epoch 175
Validation prc-auc = 0.438097
Epoch 176
Validation prc-auc = 0.436859
Epoch 177
Validation prc-auc = 0.420389
Epoch 178
Validation prc-auc = 0.411847
Epoch 179
Loss = 2.7013e-02, PNorm = 95.6189, GNorm = 0.4945, lr_0 = 7.4492e-04
Validation prc-auc = 0.440393
Epoch 180
Validation prc-auc = 0.450265
Epoch 181
Validation prc-auc = 0.450522
Epoch 182
Validation prc-auc = 0.455788
Epoch 183
Validation prc-auc = 0.439579
Epoch 184
Loss = 7.5789e-02, PNorm = 95.7485, GNorm = 2.8377, lr_0 = 7.3855e-04
Validation prc-auc = 0.440143
Epoch 185
Validation prc-auc = 0.439761
Epoch 186
Validation prc-auc = 0.431872
Epoch 187
Validation prc-auc = 0.440293
Epoch 188
Validation prc-auc = 0.441072
Epoch 189
Loss = 5.4442e-03, PNorm = 95.8673, GNorm = 0.2281, lr_0 = 7.3224e-04
Validation prc-auc = 0.440378
Epoch 190
Validation prc-auc = 0.443418
Epoch 191
Validation prc-auc = 0.468150
Epoch 192
Validation prc-auc = 0.466692
Epoch 193
Validation prc-auc = 0.457426
Epoch 194
Loss = 7.3775e-03, PNorm = 96.0325, GNorm = 0.4866, lr_0 = 7.2598e-04
Validation prc-auc = 0.456740
Epoch 195
Validation prc-auc = 0.439410
Epoch 196
Validation prc-auc = 0.478772
Epoch 197
Validation prc-auc = 0.473564
Epoch 198
Validation prc-auc = 0.436079
Epoch 199
Loss = 1.7224e-02, PNorm = 96.2280, GNorm = 0.8781, lr_0 = 7.1977e-04
Validation prc-auc = 0.409257
Epoch 200
Validation prc-auc = 0.412956
Epoch 201
Validation prc-auc = 0.415977
Epoch 202
Validation prc-auc = 0.436247
Epoch 203
Validation prc-auc = 0.429997
Epoch 204
Loss = 9.2039e-02, PNorm = 96.4660, GNorm = 2.6276, lr_0 = 7.1362e-04
Validation prc-auc = 0.398744
Epoch 205
Validation prc-auc = 0.393354
Epoch 206
Validation prc-auc = 0.389605
Epoch 207
Validation prc-auc = 0.421344
Epoch 208
Validation prc-auc = 0.443505
Epoch 209
Loss = 4.3724e-02, PNorm = 96.7233, GNorm = 1.4179, lr_0 = 7.0752e-04
Validation prc-auc = 0.434965
Epoch 210
Validation prc-auc = 0.396188
Epoch 211
Validation prc-auc = 0.359016
Epoch 212
Validation prc-auc = 0.358522
Epoch 213
Validation prc-auc = 0.415919
Epoch 214
Loss = 1.5830e-03, PNorm = 96.9388, GNorm = 0.0360, lr_0 = 7.0147e-04
Validation prc-auc = 0.396816
Epoch 215
Validation prc-auc = 0.405185
Epoch 216
Validation prc-auc = 0.412256
Epoch 217
Validation prc-auc = 0.401176
Epoch 218
Validation prc-auc = 0.440502
Epoch 219
Loss = 7.5058e-03, PNorm = 97.0994, GNorm = 0.2724, lr_0 = 6.9548e-04
Validation prc-auc = 0.430261
Epoch 220
Validation prc-auc = 0.426424
Epoch 221
Validation prc-auc = 0.431885
Epoch 222
Validation prc-auc = 0.428396
Epoch 223
Validation prc-auc = 0.417460
Epoch 224
Loss = 1.2307e-02, PNorm = 97.2308, GNorm = 0.7443, lr_0 = 6.8953e-04
Validation prc-auc = 0.434163
Epoch 225
Validation prc-auc = 0.432580
Epoch 226
Validation prc-auc = 0.419991
Epoch 227
Validation prc-auc = 0.403854
Epoch 228
Validation prc-auc = 0.428374
Epoch 229
Loss = 5.5247e-03, PNorm = 97.3274, GNorm = 0.0497, lr_0 = 6.8364e-04
Validation prc-auc = 0.428134
Epoch 230
Validation prc-auc = 0.451415
Epoch 231
Validation prc-auc = 0.453697
Epoch 232
Validation prc-auc = 0.454665
Epoch 233
Validation prc-auc = 0.455843
Epoch 234
Loss = 1.4513e-02, PNorm = 97.3863, GNorm = 0.0890, lr_0 = 6.7779e-04
Validation prc-auc = 0.457880
Epoch 235
Validation prc-auc = 0.449734
Epoch 236
Validation prc-auc = 0.443929
Epoch 237
Validation prc-auc = 0.438453
Epoch 238
Validation prc-auc = 0.436178
Epoch 239
Loss = 1.2971e-03, PNorm = 97.4494, GNorm = 0.1221, lr_0 = 6.7200e-04
Validation prc-auc = 0.433959
Epoch 240
Validation prc-auc = 0.427563
Epoch 241
Validation prc-auc = 0.416845
Epoch 242
Validation prc-auc = 0.433418
Epoch 243
Validation prc-auc = 0.424716
Epoch 244
Loss = 5.5032e-04, PNorm = 97.5011, GNorm = 0.0172, lr_0 = 6.6625e-04
Validation prc-auc = 0.437199
Epoch 245
Validation prc-auc = 0.412501
Epoch 246
Validation prc-auc = 0.412179
Epoch 247
Validation prc-auc = 0.412075
Epoch 248
Validation prc-auc = 0.414688
Epoch 249
Loss = 7.1559e-04, PNorm = 97.5379, GNorm = 0.0169, lr_0 = 6.6056e-04
Validation prc-auc = 0.421491
Epoch 250
Validation prc-auc = 0.426182
Epoch 251
Validation prc-auc = 0.433581
Epoch 252
Validation prc-auc = 0.434576
Epoch 253
Validation prc-auc = 0.434895
Epoch 254
Loss = 3.0571e-03, PNorm = 97.5615, GNorm = 0.4485, lr_0 = 6.5491e-04
Validation prc-auc = 0.434809
Epoch 255
Validation prc-auc = 0.435354
Epoch 256
Validation prc-auc = 0.415262
Epoch 257
Validation prc-auc = 0.415486
Epoch 258
Validation prc-auc = 0.415114
Epoch 259
Loss = 2.8620e-04, PNorm = 97.5761, GNorm = 0.0062, lr_0 = 6.4931e-04
Validation prc-auc = 0.422388
Epoch 260
Validation prc-auc = 0.421716
Epoch 261
Validation prc-auc = 0.423397
Epoch 262
Validation prc-auc = 0.425383
Epoch 263
Validation prc-auc = 0.435071
Epoch 264
Loss = 1.1535e-02, PNorm = 97.6106, GNorm = 0.0202, lr_0 = 6.4376e-04
Validation prc-auc = 0.436503
Epoch 265
Validation prc-auc = 0.432566
Epoch 266
Validation prc-auc = 0.446058
Epoch 267
Validation prc-auc = 0.443567
Epoch 268
Validation prc-auc = 0.454921
Epoch 269
Loss = 3.6298e-02, PNorm = 97.7809, GNorm = 2.8084, lr_0 = 6.3826e-04
Validation prc-auc = 0.441921
Epoch 270
Validation prc-auc = 0.433621
Epoch 271
Validation prc-auc = 0.407702
Epoch 272
Validation prc-auc = 0.407479
Epoch 273
Validation prc-auc = 0.413975
Epoch 274
Loss = 3.1802e-03, PNorm = 97.9253, GNorm = 0.0250, lr_0 = 6.3280e-04
Validation prc-auc = 0.417243
Epoch 275
Validation prc-auc = 0.420389
Epoch 276
Validation prc-auc = 0.423418
Epoch 277
Validation prc-auc = 0.424463
Epoch 278
Validation prc-auc = 0.421204
Epoch 279
Loss = 6.3843e-03, PNorm = 98.0520, GNorm = 0.7157, lr_0 = 6.2739e-04
Validation prc-auc = 0.419608
Epoch 280
Validation prc-auc = 0.417396
Epoch 281
Validation prc-auc = 0.444238
Epoch 282
Validation prc-auc = 0.463476
Epoch 283
Validation prc-auc = 0.457978
Epoch 284
Loss = 1.0491e-02, PNorm = 98.1595, GNorm = 0.9566, lr_0 = 6.2203e-04
Validation prc-auc = 0.454699
Epoch 285
Validation prc-auc = 0.457038
Epoch 286
Validation prc-auc = 0.441983
Epoch 287
Validation prc-auc = 0.425622
Epoch 288
Validation prc-auc = 0.405081
Epoch 289
Loss = 9.2187e-03, PNorm = 98.2719, GNorm = 0.4898, lr_0 = 6.1671e-04
Validation prc-auc = 0.406383
Epoch 290
Validation prc-auc = 0.407817
Epoch 291
Validation prc-auc = 0.405364
Epoch 292
Validation prc-auc = 0.378063
Epoch 293
Validation prc-auc = 0.391326
Epoch 294
Loss = 2.6202e-02, PNorm = 98.3753, GNorm = 1.2110, lr_0 = 6.1144e-04
Validation prc-auc = 0.398507
Epoch 295
Validation prc-auc = 0.431880
Epoch 296
Validation prc-auc = 0.422901
Epoch 297
Validation prc-auc = 0.439546
Epoch 298
Validation prc-auc = 0.428976
Epoch 299
Loss = 7.4406e-02, PNorm = 98.5759, GNorm = 0.5363, lr_0 = 6.0621e-04
Validation prc-auc = 0.428825
Model 0 best validation prc-auc = 0.478772 on epoch 196
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.253857
Ensemble test prc-auc = 0.253857
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.320942
Epoch 1
Validation prc-auc = 0.334801
Epoch 2
Validation prc-auc = 0.342303
Epoch 3
Validation prc-auc = 0.355003
Epoch 4
Loss = 5.2668e-01, PNorm = 87.5369, GNorm = 2.1603, lr_0 = 6.5000e-04
Validation prc-auc = 0.334318
Epoch 5
Validation prc-auc = 0.347826
Epoch 6
Validation prc-auc = 0.394645
Epoch 7
Validation prc-auc = 0.354888
Epoch 8
Validation prc-auc = 0.397338
Epoch 9
Loss = 4.0273e-01, PNorm = 87.9003, GNorm = 1.2058, lr_0 = 9.9743e-04
Validation prc-auc = 0.356445
Epoch 10
Validation prc-auc = 0.296336
Epoch 11
Validation prc-auc = 0.385217
Epoch 12
Validation prc-auc = 0.450850
Epoch 13
Validation prc-auc = 0.445799
Epoch 14
Loss = 3.3958e-01, PNorm = 88.2938, GNorm = 2.6004, lr_0 = 9.8890e-04
Validation prc-auc = 0.422092
Epoch 15
Validation prc-auc = 0.335748
Epoch 16
Validation prc-auc = 0.266584
Epoch 17
Validation prc-auc = 0.384970
Epoch 18
Validation prc-auc = 0.380456
Epoch 19
Loss = 3.1163e-01, PNorm = 88.6072, GNorm = 2.6732, lr_0 = 9.8045e-04
Validation prc-auc = 0.395849
Epoch 20
Validation prc-auc = 0.391947
Epoch 21
Validation prc-auc = 0.338566
Epoch 22
Validation prc-auc = 0.341552
Epoch 23
Validation prc-auc = 0.376776
Epoch 24
Loss = 3.8013e-01, PNorm = 88.8518, GNorm = 1.9058, lr_0 = 9.7207e-04
Validation prc-auc = 0.371606
Epoch 25
Validation prc-auc = 0.425878
Epoch 26
Validation prc-auc = 0.347660
Epoch 27
Validation prc-auc = 0.340710
Epoch 28
Validation prc-auc = 0.396024
Epoch 29
Loss = 2.5644e-01, PNorm = 89.0881, GNorm = 1.4875, lr_0 = 9.6376e-04
Validation prc-auc = 0.428724
Epoch 30
Validation prc-auc = 0.429486
Epoch 31
Validation prc-auc = 0.360496
Epoch 32
Validation prc-auc = 0.350406
Epoch 33
Validation prc-auc = 0.438383
Epoch 34
Loss = 3.8284e-01, PNorm = 89.3142, GNorm = 3.0969, lr_0 = 9.5552e-04
Validation prc-auc = 0.416735
Epoch 35
Validation prc-auc = 0.375840
Epoch 36
Validation prc-auc = 0.395630
Epoch 37
Validation prc-auc = 0.450016
Epoch 38
Validation prc-auc = 0.462064
Epoch 39
Loss = 3.2764e-01, PNorm = 89.5446, GNorm = 1.4377, lr_0 = 9.4735e-04
Validation prc-auc = 0.429299
Epoch 40
Validation prc-auc = 0.432247
Epoch 41
Validation prc-auc = 0.427201
Epoch 42
Validation prc-auc = 0.427025
Epoch 43
Validation prc-auc = 0.409292
Epoch 44
Loss = 3.1276e-01, PNorm = 89.7543, GNorm = 2.7344, lr_0 = 9.3925e-04
Validation prc-auc = 0.384098
Epoch 45
Validation prc-auc = 0.346920
Epoch 46
Validation prc-auc = 0.385852
Epoch 47
Validation prc-auc = 0.379080
Epoch 48
Validation prc-auc = 0.390522
Epoch 49
Loss = 2.2142e-01, PNorm = 89.9788, GNorm = 1.4726, lr_0 = 9.3122e-04
Validation prc-auc = 0.298998
Epoch 50
Validation prc-auc = 0.348693
Epoch 51
Validation prc-auc = 0.415375
Epoch 52
Validation prc-auc = 0.422324
Epoch 53
Validation prc-auc = 0.384266
Epoch 54
Loss = 2.2525e-01, PNorm = 90.2085, GNorm = 1.6548, lr_0 = 9.2326e-04
Validation prc-auc = 0.326057
Epoch 55
Validation prc-auc = 0.298286
Epoch 56
Validation prc-auc = 0.373380
Epoch 57
Validation prc-auc = 0.407054
Epoch 58
Validation prc-auc = 0.425260
Epoch 59
Loss = 1.0932e-01, PNorm = 90.4530, GNorm = 0.8842, lr_0 = 9.1537e-04
Validation prc-auc = 0.430609
Epoch 60
Validation prc-auc = 0.411402
Epoch 61
Validation prc-auc = 0.379658
Epoch 62
Validation prc-auc = 0.399708
Epoch 63
Validation prc-auc = 0.430999
Epoch 64
Loss = 1.2426e-01, PNorm = 90.7117, GNorm = 1.3833, lr_0 = 9.0754e-04
Validation prc-auc = 0.433632
Epoch 65
Validation prc-auc = 0.427959
Epoch 66
Validation prc-auc = 0.429855
Epoch 67
Validation prc-auc = 0.368519
Epoch 68
Validation prc-auc = 0.438212
Epoch 69
Loss = 7.3615e-02, PNorm = 90.9625, GNorm = 0.6808, lr_0 = 8.9978e-04
Validation prc-auc = 0.455287
Epoch 70
Validation prc-auc = 0.459904
Epoch 71
Validation prc-auc = 0.443168
Epoch 72
Validation prc-auc = 0.431872
Epoch 73
Validation prc-auc = 0.438937
Epoch 74
Loss = 1.0633e-01, PNorm = 91.2300, GNorm = 0.8058, lr_0 = 8.9209e-04
Validation prc-auc = 0.443091
Epoch 75
Validation prc-auc = 0.405697
Epoch 76
Validation prc-auc = 0.387005
Epoch 77
Validation prc-auc = 0.411594
Epoch 78
Validation prc-auc = 0.415083
Epoch 79
Loss = 1.3720e-01, PNorm = 91.5141, GNorm = 0.8861, lr_0 = 8.8447e-04
Validation prc-auc = 0.395491
Epoch 80
Validation prc-auc = 0.224115
Epoch 81
Validation prc-auc = 0.462860
Epoch 82
Validation prc-auc = 0.456340
Epoch 83
Validation prc-auc = 0.434970
Epoch 84
Loss = 1.3687e-01, PNorm = 91.8360, GNorm = 2.6296, lr_0 = 8.7691e-04
Validation prc-auc = 0.425731
Epoch 85
Validation prc-auc = 0.462399
Epoch 86
Validation prc-auc = 0.397852
Epoch 87
Validation prc-auc = 0.300591
Epoch 88
Validation prc-auc = 0.258600
Epoch 89
Loss = 1.2352e-01, PNorm = 92.1746, GNorm = 0.9679, lr_0 = 8.6941e-04
Validation prc-auc = 0.260755
Epoch 90
Validation prc-auc = 0.255680
Epoch 91
Validation prc-auc = 0.416923
Epoch 92
Validation prc-auc = 0.413610
Epoch 93
Validation prc-auc = 0.376678
Epoch 94
Loss = 2.8709e-02, PNorm = 92.4776, GNorm = 0.3184, lr_0 = 8.6198e-04
Validation prc-auc = 0.366269
Epoch 95
Validation prc-auc = 0.430128
Epoch 96
Validation prc-auc = 0.438681
Epoch 97
Validation prc-auc = 0.418945
Epoch 98
Validation prc-auc = 0.430879
Epoch 99
Loss = 8.4377e-02, PNorm = 92.7628, GNorm = 0.6778, lr_0 = 8.5461e-04
Validation prc-auc = 0.467257
Epoch 100
Validation prc-auc = 0.474843
Epoch 101
Validation prc-auc = 0.463863
Epoch 102
Validation prc-auc = 0.290841
Epoch 103
Validation prc-auc = 0.285283
Epoch 104
Loss = 1.7662e-01, PNorm = 93.0620, GNorm = 0.9088, lr_0 = 8.4730e-04
Validation prc-auc = 0.393924
Epoch 105
Validation prc-auc = 0.413038
Epoch 106
Validation prc-auc = 0.427298
Epoch 107
Validation prc-auc = 0.430084
Epoch 108
Validation prc-auc = 0.441626
Epoch 109
Loss = 1.0112e-02, PNorm = 93.3380, GNorm = 0.1249, lr_0 = 8.4006e-04
Validation prc-auc = 0.446201
Epoch 110
Validation prc-auc = 0.444113
Epoch 111
Validation prc-auc = 0.461839
Epoch 112
Validation prc-auc = 0.442928
Epoch 113
Validation prc-auc = 0.409151
Epoch 114
Loss = 7.0014e-02, PNorm = 93.5474, GNorm = 0.3794, lr_0 = 8.3288e-04
Validation prc-auc = 0.399162
Epoch 115
Validation prc-auc = 0.435878
Epoch 116
Validation prc-auc = 0.463409
Epoch 117
Validation prc-auc = 0.448695
Epoch 118
Validation prc-auc = 0.439211
Epoch 119
Loss = 1.6966e-01, PNorm = 93.7264, GNorm = 1.8348, lr_0 = 8.2576e-04
Validation prc-auc = 0.433712
Epoch 120
Validation prc-auc = 0.431398
Epoch 121
Validation prc-auc = 0.445772
Epoch 122
Validation prc-auc = 0.432490
Epoch 123
Validation prc-auc = 0.420042
Epoch 124
Loss = 5.1855e-02, PNorm = 93.9650, GNorm = 0.7062, lr_0 = 8.1870e-04
Validation prc-auc = 0.416945
Epoch 125
Validation prc-auc = 0.435560
Epoch 126
Validation prc-auc = 0.450043
Epoch 127
Validation prc-auc = 0.450124
Epoch 128
Validation prc-auc = 0.461018
Epoch 129
Loss = 4.8951e-02, PNorm = 94.1571, GNorm = 0.7424, lr_0 = 8.1170e-04
Validation prc-auc = 0.459135
Epoch 130
Validation prc-auc = 0.446988
Epoch 131
Validation prc-auc = 0.444195
Epoch 132
Validation prc-auc = 0.449650
Epoch 133
Validation prc-auc = 0.456067
Epoch 134
Loss = 5.5184e-02, PNorm = 94.3031, GNorm = 0.5037, lr_0 = 8.0476e-04
Validation prc-auc = 0.460114
Epoch 135
Validation prc-auc = 0.464014
Epoch 136
Validation prc-auc = 0.452721
Epoch 137
Validation prc-auc = 0.453428
Epoch 138
Validation prc-auc = 0.451669
Epoch 139
Loss = 4.9141e-03, PNorm = 94.4184, GNorm = 0.2672, lr_0 = 7.9788e-04
Validation prc-auc = 0.458928
Epoch 140
Validation prc-auc = 0.455854
Epoch 141
Validation prc-auc = 0.454810
Epoch 142
Validation prc-auc = 0.456332
Epoch 143
Validation prc-auc = 0.456999
Epoch 144
Loss = 3.2250e-02, PNorm = 94.5210, GNorm = 0.9324, lr_0 = 7.9106e-04
Validation prc-auc = 0.456175
Epoch 145
Validation prc-auc = 0.457555
Epoch 146
Validation prc-auc = 0.436580
Epoch 147
Validation prc-auc = 0.434179
Epoch 148
Validation prc-auc = 0.436116
Epoch 149
Loss = 2.5283e-02, PNorm = 94.6390, GNorm = 0.0228, lr_0 = 7.8430e-04
Validation prc-auc = 0.434899
Epoch 150
Validation prc-auc = 0.433729
Epoch 151
Validation prc-auc = 0.432342
Epoch 152
Validation prc-auc = 0.448069
Epoch 153
Validation prc-auc = 0.467857
Epoch 154
Loss = 5.2351e-03, PNorm = 94.7877, GNorm = 0.0643, lr_0 = 7.7759e-04
Validation prc-auc = 0.466573
Epoch 155
Validation prc-auc = 0.453815
Epoch 156
Validation prc-auc = 0.452952
Epoch 157
Validation prc-auc = 0.464149
Epoch 158
Validation prc-auc = 0.466485
Epoch 159
Loss = 2.3925e-02, PNorm = 94.9038, GNorm = 1.3638, lr_0 = 7.7095e-04
Validation prc-auc = 0.465648
Epoch 160
Validation prc-auc = 0.461725
Epoch 161
Validation prc-auc = 0.460464
Epoch 162
Validation prc-auc = 0.460477
Epoch 163
Validation prc-auc = 0.448746
Epoch 164
Loss = 5.6064e-02, PNorm = 95.0673, GNorm = 1.4231, lr_0 = 7.6436e-04
Validation prc-auc = 0.445227
Epoch 165
Validation prc-auc = 0.427134
Epoch 166
Validation prc-auc = 0.432037
Epoch 167
Validation prc-auc = 0.425341
Epoch 168
Validation prc-auc = 0.435517
Epoch 169
Loss = 1.4915e-02, PNorm = 95.2959, GNorm = 0.4009, lr_0 = 7.5782e-04
Validation prc-auc = 0.423889
Epoch 170
Validation prc-auc = 0.431333
Epoch 171
Validation prc-auc = 0.429438
Epoch 172
Validation prc-auc = 0.435144
Epoch 173
Validation prc-auc = 0.429392
Epoch 174
Loss = 5.6106e-02, PNorm = 95.4955, GNorm = 1.8612, lr_0 = 7.5134e-04
Validation prc-auc = 0.429006
Epoch 175
Validation prc-auc = 0.442039
Epoch 176
Validation prc-auc = 0.452854
Epoch 177
Validation prc-auc = 0.456716
Epoch 178
Validation prc-auc = 0.457036
Epoch 179
Loss = 7.2267e-03, PNorm = 95.7039, GNorm = 0.3254, lr_0 = 7.4492e-04
Validation prc-auc = 0.449290
Epoch 180
Validation prc-auc = 0.455219
Epoch 181
Validation prc-auc = 0.455136
Epoch 182
Validation prc-auc = 0.450797
Epoch 183
Validation prc-auc = 0.448871
Epoch 184
Loss = 1.0854e-01, PNorm = 95.8615, GNorm = 2.4374, lr_0 = 7.3855e-04
Validation prc-auc = 0.446606
Epoch 185
Validation prc-auc = 0.439141
Epoch 186
Validation prc-auc = 0.439334
Epoch 187
Validation prc-auc = 0.407350
Epoch 188
Validation prc-auc = 0.377428
Epoch 189
Loss = 1.6323e-01, PNorm = 96.0405, GNorm = 2.3953, lr_0 = 7.3224e-04
Validation prc-auc = 0.417929
Epoch 190
Validation prc-auc = 0.441970
Epoch 191
Validation prc-auc = 0.455563
Epoch 192
Validation prc-auc = 0.430198
Epoch 193
Validation prc-auc = 0.416963
Epoch 194
Loss = 6.4502e-02, PNorm = 96.3220, GNorm = 1.0244, lr_0 = 7.2598e-04
Validation prc-auc = 0.397650
Epoch 195
Validation prc-auc = 0.417855
Epoch 196
Validation prc-auc = 0.432669
Epoch 197
Validation prc-auc = 0.439481
Epoch 198
Validation prc-auc = 0.441681
Epoch 199
Loss = 3.4519e-02, PNorm = 96.5818, GNorm = 1.4691, lr_0 = 7.1977e-04
Validation prc-auc = 0.448405
Epoch 200
Validation prc-auc = 0.440337
Epoch 201
Validation prc-auc = 0.426529
Epoch 202
Validation prc-auc = 0.419161
Epoch 203
Validation prc-auc = 0.413939
Epoch 204
Loss = 1.4335e-02, PNorm = 96.7471, GNorm = 0.0476, lr_0 = 7.1362e-04
Validation prc-auc = 0.396132
Epoch 205
Validation prc-auc = 0.390842
Epoch 206
Validation prc-auc = 0.412918
Epoch 207
Validation prc-auc = 0.414273
Epoch 208
Validation prc-auc = 0.415932
Epoch 209
Loss = 4.0178e-02, PNorm = 96.8571, GNorm = 0.9103, lr_0 = 7.0752e-04
Validation prc-auc = 0.418492
Epoch 210
Validation prc-auc = 0.424332
Epoch 211
Validation prc-auc = 0.432068
Epoch 212
Validation prc-auc = 0.432774
Epoch 213
Validation prc-auc = 0.437472
Epoch 214
Loss = 2.9550e-03, PNorm = 96.9330, GNorm = 0.0795, lr_0 = 7.0147e-04
Validation prc-auc = 0.438522
Epoch 215
Validation prc-auc = 0.437477
Epoch 216
Validation prc-auc = 0.436658
Epoch 217
Validation prc-auc = 0.436898
Epoch 218
Validation prc-auc = 0.438667
Epoch 219
Loss = 1.4140e-02, PNorm = 96.9943, GNorm = 0.7065, lr_0 = 6.9548e-04
Validation prc-auc = 0.433291
Epoch 220
Validation prc-auc = 0.428902
Epoch 221
Validation prc-auc = 0.428125
Epoch 222
Validation prc-auc = 0.435993
Epoch 223
Validation prc-auc = 0.441728
Epoch 224
Loss = 6.4733e-03, PNorm = 97.0400, GNorm = 0.1172, lr_0 = 6.8953e-04
Validation prc-auc = 0.441921
Epoch 225
Validation prc-auc = 0.443037
Epoch 226
Validation prc-auc = 0.442701
Epoch 227
Validation prc-auc = 0.435715
Epoch 228
Validation prc-auc = 0.429828
Epoch 229
Loss = 1.8792e-03, PNorm = 97.0760, GNorm = 0.1567, lr_0 = 6.8364e-04
Validation prc-auc = 0.435681
Epoch 230
Validation prc-auc = 0.451400
Epoch 231
Validation prc-auc = 0.453611
Epoch 232
Validation prc-auc = 0.449329
Epoch 233
Validation prc-auc = 0.436043
Epoch 234
Loss = 2.1065e-02, PNorm = 97.1497, GNorm = 0.0937, lr_0 = 6.7779e-04
Validation prc-auc = 0.451700
Epoch 235
Validation prc-auc = 0.452337
Epoch 236
Validation prc-auc = 0.441392
Epoch 237
Validation prc-auc = 0.427028
Epoch 238
Validation prc-auc = 0.427852
Epoch 239
Loss = 9.8020e-02, PNorm = 97.3061, GNorm = 1.2643, lr_0 = 6.7200e-04
Validation prc-auc = 0.453287
Epoch 240
Validation prc-auc = 0.465334
Epoch 241
Validation prc-auc = 0.470776
Epoch 242
Validation prc-auc = 0.460844
Epoch 243
Validation prc-auc = 0.463815
Epoch 244
Loss = 2.1879e-02, PNorm = 97.5636, GNorm = 0.6265, lr_0 = 6.6625e-04
Validation prc-auc = 0.455728
Epoch 245
Validation prc-auc = 0.460307
Epoch 246
Validation prc-auc = 0.454270
Epoch 247
Validation prc-auc = 0.464582
Epoch 248
Validation prc-auc = 0.459229
Epoch 249
Loss = 2.0907e-02, PNorm = 97.7722, GNorm = 0.2557, lr_0 = 6.6056e-04
Validation prc-auc = 0.452843
Epoch 250
Validation prc-auc = 0.440207
Epoch 251
Validation prc-auc = 0.442129
Epoch 252
Validation prc-auc = 0.441899
Epoch 253
Validation prc-auc = 0.420650
Epoch 254
Loss = 9.6995e-03, PNorm = 97.9806, GNorm = 0.5605, lr_0 = 6.5491e-04
Validation prc-auc = 0.396836
Epoch 255
Validation prc-auc = 0.372582
Epoch 256
Validation prc-auc = 0.375067
Epoch 257
Validation prc-auc = 0.405112
Epoch 258
Validation prc-auc = 0.412370
Epoch 259
Loss = 1.5769e-02, PNorm = 98.2410, GNorm = 0.5293, lr_0 = 6.4931e-04
Validation prc-auc = 0.424524
Epoch 260
Validation prc-auc = 0.434939
Epoch 261
Validation prc-auc = 0.443382
Epoch 262
Validation prc-auc = 0.445475
Epoch 263
Validation prc-auc = 0.441224
Epoch 264
Loss = 2.7155e-02, PNorm = 98.4455, GNorm = 0.3404, lr_0 = 6.4376e-04
Validation prc-auc = 0.441226
Epoch 265
Validation prc-auc = 0.435171
Epoch 266
Validation prc-auc = 0.427360
Epoch 267
Validation prc-auc = 0.425935
Epoch 268
Validation prc-auc = 0.427284
Epoch 269
Loss = 2.7117e-03, PNorm = 98.5788, GNorm = 0.1214, lr_0 = 6.3826e-04
Validation prc-auc = 0.428077
Epoch 270
Validation prc-auc = 0.428711
Epoch 271
Validation prc-auc = 0.446212
Epoch 272
Validation prc-auc = 0.448541
Epoch 273
Validation prc-auc = 0.439043
Epoch 274
Loss = 2.1056e-03, PNorm = 98.6605, GNorm = 0.0910, lr_0 = 6.3280e-04
Validation prc-auc = 0.440862
Epoch 275
Validation prc-auc = 0.446808
Epoch 276
Validation prc-auc = 0.442976
Epoch 277
Validation prc-auc = 0.451163
Epoch 278
Validation prc-auc = 0.449076
Epoch 279
Loss = 3.9689e-02, PNorm = 98.7126, GNorm = 1.9213, lr_0 = 6.2739e-04
Validation prc-auc = 0.445602
Epoch 280
Validation prc-auc = 0.435803
Epoch 281
Validation prc-auc = 0.434941
Epoch 282
Validation prc-auc = 0.433852
Epoch 283
Validation prc-auc = 0.428252
Epoch 284
Loss = 1.7113e-02, PNorm = 98.7743, GNorm = 2.0348, lr_0 = 6.2203e-04
Validation prc-auc = 0.418143
Epoch 285
Validation prc-auc = 0.428196
Epoch 286
Validation prc-auc = 0.431735
Epoch 287
Validation prc-auc = 0.438944
Epoch 288
Validation prc-auc = 0.427029
Epoch 289
Loss = 3.5198e-03, PNorm = 98.8542, GNorm = 0.1627, lr_0 = 6.1671e-04
Validation prc-auc = 0.431806
Epoch 290
Validation prc-auc = 0.441142
Epoch 291
Validation prc-auc = 0.442670
Epoch 292
Validation prc-auc = 0.454172
Epoch 293
Validation prc-auc = 0.453523
Epoch 294
Loss = 1.3689e-03, PNorm = 98.9158, GNorm = 0.0218, lr_0 = 6.1144e-04
Validation prc-auc = 0.463935
Epoch 295
Validation prc-auc = 0.464718
Epoch 296
Validation prc-auc = 0.466295
Epoch 297
Validation prc-auc = 0.468292
Epoch 298
Validation prc-auc = 0.469508
Epoch 299
Loss = 4.8500e-03, PNorm = 98.9422, GNorm = 0.3874, lr_0 = 6.0621e-04
Validation prc-auc = 0.474055
Model 0 best validation prc-auc = 0.474843 on epoch 100
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.320228
Ensemble test prc-auc = 0.320228
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.406327
Epoch 1
Validation prc-auc = 0.437636
Epoch 2
Validation prc-auc = 0.381028
Epoch 3
Validation prc-auc = 0.180187
Epoch 4
Loss = 6.6223e-01, PNorm = 87.5393, GNorm = 2.0274, lr_0 = 6.5000e-04
Validation prc-auc = 0.357518
Epoch 5
Validation prc-auc = 0.433536
Epoch 6
Validation prc-auc = 0.426368
Epoch 7
Validation prc-auc = 0.404317
Epoch 8
Validation prc-auc = 0.346010
Epoch 9
Loss = 5.0566e-01, PNorm = 87.8985, GNorm = 1.9844, lr_0 = 9.9743e-04
Validation prc-auc = 0.284407
Epoch 10
Validation prc-auc = 0.398992
Epoch 11
Validation prc-auc = 0.423578
Epoch 12
Validation prc-auc = 0.388518
Epoch 13
Validation prc-auc = 0.305293
Epoch 14
Loss = 3.4932e-01, PNorm = 88.2846, GNorm = 2.4945, lr_0 = 9.8890e-04
Validation prc-auc = 0.336871
Epoch 15
Validation prc-auc = 0.410964
Epoch 16
Validation prc-auc = 0.427583
Epoch 17
Validation prc-auc = 0.359288
Epoch 18
Validation prc-auc = 0.306847
Epoch 19
Loss = 2.3845e-01, PNorm = 88.5708, GNorm = 1.0482, lr_0 = 9.8045e-04
Validation prc-auc = 0.347365
Epoch 20
Validation prc-auc = 0.389584
Epoch 21
Validation prc-auc = 0.398092
Epoch 22
Validation prc-auc = 0.322625
Epoch 23
Validation prc-auc = 0.327867
Epoch 24
Loss = 3.9150e-01, PNorm = 88.8088, GNorm = 1.4835, lr_0 = 9.7207e-04
Validation prc-auc = 0.399702
Epoch 25
Validation prc-auc = 0.368330
Epoch 26
Validation prc-auc = 0.175833
Epoch 27
Validation prc-auc = 0.221044
Epoch 28
Validation prc-auc = 0.426231
Epoch 29
Loss = 4.0391e-01, PNorm = 89.0552, GNorm = 1.7840, lr_0 = 9.6376e-04
Validation prc-auc = 0.449471
Epoch 30
Validation prc-auc = 0.454006
Epoch 31
Validation prc-auc = 0.428286
Epoch 32
Validation prc-auc = 0.415073
Epoch 33
Validation prc-auc = 0.371844
Epoch 34
Loss = 2.7212e-01, PNorm = 89.3073, GNorm = 1.0704, lr_0 = 9.5552e-04
Validation prc-auc = 0.349336
Epoch 35
Validation prc-auc = 0.331412
Epoch 36
Validation prc-auc = 0.288945
Epoch 37
Validation prc-auc = 0.344321
Epoch 38
Validation prc-auc = 0.336327
Epoch 39
Loss = 1.8403e-01, PNorm = 89.5662, GNorm = 1.2783, lr_0 = 9.4735e-04
Validation prc-auc = 0.273701
Epoch 40
Validation prc-auc = 0.276317
Epoch 41
Validation prc-auc = 0.399069
Epoch 42
Validation prc-auc = 0.402477
Epoch 43
Validation prc-auc = 0.339910
Epoch 44
Loss = 2.6964e-01, PNorm = 89.7788, GNorm = 2.4721, lr_0 = 9.3925e-04
Validation prc-auc = 0.410704
Epoch 45
Validation prc-auc = 0.405317
Epoch 46
Validation prc-auc = 0.418850
Epoch 47
Validation prc-auc = 0.422858
Epoch 48
Validation prc-auc = 0.420989
Epoch 49
Loss = 1.4962e-01, PNorm = 90.0177, GNorm = 0.7753, lr_0 = 9.3122e-04
Validation prc-auc = 0.441014
Epoch 50
Validation prc-auc = 0.446325
Epoch 51
Validation prc-auc = 0.448065
Epoch 52
Validation prc-auc = 0.441941
Epoch 53
Validation prc-auc = 0.454776
Epoch 54
Loss = 1.2702e-01, PNorm = 90.2674, GNorm = 1.1951, lr_0 = 9.2326e-04
Validation prc-auc = 0.436259
Epoch 55
Validation prc-auc = 0.468799
Epoch 56
Validation prc-auc = 0.454231
Epoch 57
Validation prc-auc = 0.435403
Epoch 58
Validation prc-auc = 0.411551
Epoch 59
Loss = 1.6858e-01, PNorm = 90.5426, GNorm = 1.3138, lr_0 = 9.1537e-04
Validation prc-auc = 0.408887
Epoch 60
Validation prc-auc = 0.413830
Epoch 61
Validation prc-auc = 0.419200
Epoch 62
Validation prc-auc = 0.406473
Epoch 63
Validation prc-auc = 0.443609
Epoch 64
Loss = 1.7052e-01, PNorm = 90.8241, GNorm = 1.3697, lr_0 = 9.0754e-04
Validation prc-auc = 0.469723
Epoch 65
Validation prc-auc = 0.477077
Epoch 66
Validation prc-auc = 0.457557
Epoch 67
Validation prc-auc = 0.462509
Epoch 68
Validation prc-auc = 0.449636
Epoch 69
Loss = 7.6686e-02, PNorm = 91.1398, GNorm = 0.7111, lr_0 = 8.9978e-04
Validation prc-auc = 0.431040
Epoch 70
Validation prc-auc = 0.433881
Epoch 71
Validation prc-auc = 0.426844
Epoch 72
Validation prc-auc = 0.434131
Epoch 73
Validation prc-auc = 0.445135
Epoch 74
Loss = 7.4143e-02, PNorm = 91.4336, GNorm = 0.9702, lr_0 = 8.9209e-04
Validation prc-auc = 0.427473
Epoch 75
Validation prc-auc = 0.417753
Epoch 76
Validation prc-auc = 0.429353
Epoch 77
Validation prc-auc = 0.447524
Epoch 78
Validation prc-auc = 0.414851
Epoch 79
Loss = 1.1333e-01, PNorm = 91.6804, GNorm = 1.4158, lr_0 = 8.8447e-04
Validation prc-auc = 0.447013
Epoch 80
Validation prc-auc = 0.443864
Epoch 81
Validation prc-auc = 0.437394
Epoch 82
Validation prc-auc = 0.421019
Epoch 83
Validation prc-auc = 0.424063
Epoch 84
Loss = 1.1857e-01, PNorm = 91.9278, GNorm = 1.0763, lr_0 = 8.7691e-04
Validation prc-auc = 0.440127
Epoch 85
Validation prc-auc = 0.450079
Epoch 86
Validation prc-auc = 0.458650
Epoch 87
Validation prc-auc = 0.455711
Epoch 88
Validation prc-auc = 0.440632
Epoch 89
Loss = 1.3513e-01, PNorm = 92.2005, GNorm = 1.7740, lr_0 = 8.6941e-04
Validation prc-auc = 0.448896
Epoch 90
Validation prc-auc = 0.431929
Epoch 91
Validation prc-auc = 0.436224
Epoch 92
Validation prc-auc = 0.412948
Epoch 93
Validation prc-auc = 0.429152
Epoch 94
Loss = 1.0087e-01, PNorm = 92.4941, GNorm = 1.7562, lr_0 = 8.6198e-04
Validation prc-auc = 0.427154
Epoch 95
Validation prc-auc = 0.443868
Epoch 96
Validation prc-auc = 0.445309
Epoch 97
Validation prc-auc = 0.446334
Epoch 98
Validation prc-auc = 0.455344
Epoch 99
Loss = 8.8112e-02, PNorm = 92.7793, GNorm = 1.4147, lr_0 = 8.5461e-04
Validation prc-auc = 0.453663
Epoch 100
Validation prc-auc = 0.453911
Epoch 101
Validation prc-auc = 0.438281
Epoch 102
Validation prc-auc = 0.442699
Epoch 103
Validation prc-auc = 0.444698
Epoch 104
Loss = 5.6789e-02, PNorm = 93.0416, GNorm = 1.3456, lr_0 = 8.4730e-04
Validation prc-auc = 0.436066
Epoch 105
Validation prc-auc = 0.435525
Epoch 106
Validation prc-auc = 0.421340
Epoch 107
Validation prc-auc = 0.423399
Epoch 108
Validation prc-auc = 0.433110
Epoch 109
Loss = 1.6796e-01, PNorm = 93.3240, GNorm = 2.6943, lr_0 = 8.4006e-04
Validation prc-auc = 0.430434
Epoch 110
Validation prc-auc = 0.427431
Epoch 111
Validation prc-auc = 0.433743
Epoch 112
Validation prc-auc = 0.435313
Epoch 113
Validation prc-auc = 0.434714
Epoch 114
Loss = 1.5025e-01, PNorm = 93.5968, GNorm = 0.6263, lr_0 = 8.3288e-04
Validation prc-auc = 0.429147
Epoch 115
Validation prc-auc = 0.447944
Epoch 116
Validation prc-auc = 0.447280
Epoch 117
Validation prc-auc = 0.455742
Epoch 118
Validation prc-auc = 0.446134
Epoch 119
Loss = 4.4078e-02, PNorm = 93.8577, GNorm = 0.6204, lr_0 = 8.2576e-04
Validation prc-auc = 0.450922
Epoch 120
Validation prc-auc = 0.404482
Epoch 121
Validation prc-auc = 0.402167
Epoch 122
Validation prc-auc = 0.388909
Epoch 123
Validation prc-auc = 0.347824
Epoch 124
Loss = 8.1857e-02, PNorm = 94.0843, GNorm = 2.3772, lr_0 = 8.1870e-04
Validation prc-auc = 0.375682
Epoch 125
Validation prc-auc = 0.439169
Epoch 126
Validation prc-auc = 0.444194
Epoch 127
Validation prc-auc = 0.436782
Epoch 128
Validation prc-auc = 0.419430
Epoch 129
Loss = 6.2656e-02, PNorm = 94.2968, GNorm = 0.2348, lr_0 = 8.1170e-04
Validation prc-auc = 0.422598
Epoch 130
Validation prc-auc = 0.436139
Epoch 131
Validation prc-auc = 0.443521
Epoch 132
Validation prc-auc = 0.443721
Epoch 133
Validation prc-auc = 0.435735
Epoch 134
Loss = 2.8181e-02, PNorm = 94.5410, GNorm = 1.0431, lr_0 = 8.0476e-04
Validation prc-auc = 0.447635
Epoch 135
Validation prc-auc = 0.441464
Epoch 136
Validation prc-auc = 0.421229
Epoch 137
Validation prc-auc = 0.425668
Epoch 138
Validation prc-auc = 0.417811
Epoch 139
Loss = 5.4712e-02, PNorm = 94.7438, GNorm = 0.9272, lr_0 = 7.9788e-04
Validation prc-auc = 0.412856
Epoch 140
Validation prc-auc = 0.406761
Epoch 141
Validation prc-auc = 0.417404
Epoch 142
Validation prc-auc = 0.407545
Epoch 143
Validation prc-auc = 0.395729
Epoch 144
Loss = 1.6600e-01, PNorm = 94.9749, GNorm = 2.8903, lr_0 = 7.9106e-04
Validation prc-auc = 0.279811
Epoch 145
Validation prc-auc = 0.297061
Epoch 146
Validation prc-auc = 0.310956
Epoch 147
Validation prc-auc = 0.315112
Epoch 148
Validation prc-auc = 0.310140
Epoch 149
Loss = 1.2542e-01, PNorm = 95.1928, GNorm = 2.9882, lr_0 = 7.8430e-04
Validation prc-auc = 0.405874
Epoch 150
Validation prc-auc = 0.451108
Epoch 151
Validation prc-auc = 0.455889
Epoch 152
Validation prc-auc = 0.446191
Epoch 153
Validation prc-auc = 0.441291
Epoch 154
Loss = 1.1826e-01, PNorm = 95.4609, GNorm = 0.4969, lr_0 = 7.7759e-04
Validation prc-auc = 0.428892
Epoch 155
Validation prc-auc = 0.411415
Epoch 156
Validation prc-auc = 0.417655
Epoch 157
Validation prc-auc = 0.426287
Epoch 158
Validation prc-auc = 0.411084
Epoch 159
Loss = 2.9997e-02, PNorm = 95.7276, GNorm = 0.8158, lr_0 = 7.7095e-04
Validation prc-auc = 0.419139
Epoch 160
Validation prc-auc = 0.431571
Epoch 161
Validation prc-auc = 0.414324
Epoch 162
Validation prc-auc = 0.405982
Epoch 163
Validation prc-auc = 0.401818
Epoch 164
Loss = 2.2268e-02, PNorm = 95.9389, GNorm = 0.9992, lr_0 = 7.6436e-04
Validation prc-auc = 0.401142
Epoch 165
Validation prc-auc = 0.407158
Epoch 166
Validation prc-auc = 0.405444
Epoch 167
Validation prc-auc = 0.407069
Epoch 168
Validation prc-auc = 0.407344
Epoch 169
Loss = 2.8725e-02, PNorm = 96.0976, GNorm = 0.8869, lr_0 = 7.5782e-04
Validation prc-auc = 0.411873
Epoch 170
Validation prc-auc = 0.411236
Epoch 171
Validation prc-auc = 0.392655
Epoch 172
Validation prc-auc = 0.404800
Epoch 173
Validation prc-auc = 0.403455
Epoch 174
Loss = 5.7564e-02, PNorm = 96.2426, GNorm = 0.3642, lr_0 = 7.5134e-04
Validation prc-auc = 0.414247
Epoch 175
Validation prc-auc = 0.434987
Epoch 176
Validation prc-auc = 0.436362
Epoch 177
Validation prc-auc = 0.443894
Epoch 178
Validation prc-auc = 0.445593
Epoch 179
Loss = 2.8579e-02, PNorm = 96.3655, GNorm = 0.2505, lr_0 = 7.4492e-04
Validation prc-auc = 0.448253
Epoch 180
Validation prc-auc = 0.453422
Epoch 181
Validation prc-auc = 0.436883
Epoch 182
Validation prc-auc = 0.433016
Epoch 183
Validation prc-auc = 0.436931
Epoch 184
Loss = 6.4468e-03, PNorm = 96.4869, GNorm = 0.2150, lr_0 = 7.3855e-04
Validation prc-auc = 0.436978
Epoch 185
Validation prc-auc = 0.430086
Epoch 186
Validation prc-auc = 0.426525
Epoch 187
Validation prc-auc = 0.427154
Epoch 188
Validation prc-auc = 0.420056
Epoch 189
Loss = 1.6643e-02, PNorm = 96.5795, GNorm = 0.5099, lr_0 = 7.3224e-04
Validation prc-auc = 0.418958
Epoch 190
Validation prc-auc = 0.424505
Epoch 191
Validation prc-auc = 0.423611
Epoch 192
Validation prc-auc = 0.420916
Epoch 193
Validation prc-auc = 0.434123
Epoch 194
Loss = 2.3704e-02, PNorm = 96.6729, GNorm = 0.7243, lr_0 = 7.2598e-04
Validation prc-auc = 0.424925
Epoch 195
Validation prc-auc = 0.425809
Epoch 196
Validation prc-auc = 0.429397
Epoch 197
Validation prc-auc = 0.419871
Epoch 198
Validation prc-auc = 0.429235
Epoch 199
Loss = 3.0165e-02, PNorm = 96.7612, GNorm = 1.2162, lr_0 = 7.1977e-04
Validation prc-auc = 0.434576
Epoch 200
Validation prc-auc = 0.435769
Epoch 201
Validation prc-auc = 0.434736
Epoch 202
Validation prc-auc = 0.420995
Epoch 203
Validation prc-auc = 0.412459
Epoch 204
Loss = 2.5048e-02, PNorm = 96.9628, GNorm = 0.9442, lr_0 = 7.1362e-04
Validation prc-auc = 0.447120
Epoch 205
Validation prc-auc = 0.434864
Epoch 206
Validation prc-auc = 0.435530
Epoch 207
Validation prc-auc = 0.443949
Epoch 208
Validation prc-auc = 0.440016
Epoch 209
Loss = 3.9677e-03, PNorm = 97.1574, GNorm = 0.2062, lr_0 = 7.0752e-04
Validation prc-auc = 0.448506
Epoch 210
Validation prc-auc = 0.427360
Epoch 211
Validation prc-auc = 0.417238
Epoch 212
Validation prc-auc = 0.417615
Epoch 213
Validation prc-auc = 0.417935
Epoch 214
Loss = 1.8829e-02, PNorm = 97.3018, GNorm = 0.1942, lr_0 = 7.0147e-04
Validation prc-auc = 0.437379
Epoch 215
Validation prc-auc = 0.437069
Epoch 216
Validation prc-auc = 0.436634
Epoch 217
Validation prc-auc = 0.437857
Epoch 218
Validation prc-auc = 0.436806
Epoch 219
Loss = 3.0513e-03, PNorm = 97.3807, GNorm = 0.0645, lr_0 = 6.9548e-04
Validation prc-auc = 0.436676
Epoch 220
Validation prc-auc = 0.437420
Epoch 221
Validation prc-auc = 0.429884
Epoch 222
Validation prc-auc = 0.428746
Epoch 223
Validation prc-auc = 0.420379
Epoch 224
Loss = 6.7642e-03, PNorm = 97.4433, GNorm = 0.3031, lr_0 = 6.8953e-04
Validation prc-auc = 0.434907
Epoch 225
Validation prc-auc = 0.434938
Epoch 226
Validation prc-auc = 0.419675
Epoch 227
Validation prc-auc = 0.426023
Epoch 228
Validation prc-auc = 0.430746
Epoch 229
Loss = 2.4326e-02, PNorm = 97.5007, GNorm = 1.2407, lr_0 = 6.8364e-04
Validation prc-auc = 0.429857
Epoch 230
Validation prc-auc = 0.439029
Epoch 231
Validation prc-auc = 0.438838
Epoch 232
Validation prc-auc = 0.431334
Epoch 233
Validation prc-auc = 0.432577
Epoch 234
Loss = 2.4541e-02, PNorm = 97.5578, GNorm = 1.9800, lr_0 = 6.7779e-04
Validation prc-auc = 0.435320
Epoch 235
Validation prc-auc = 0.437758
Epoch 236
Validation prc-auc = 0.439624
Epoch 237
Validation prc-auc = 0.439210
Epoch 238
Validation prc-auc = 0.440395
Epoch 239
Loss = 5.4653e-03, PNorm = 97.6494, GNorm = 0.2198, lr_0 = 6.7200e-04
Validation prc-auc = 0.427545
Epoch 240
Validation prc-auc = 0.432802
Epoch 241
Validation prc-auc = 0.438138
Epoch 242
Validation prc-auc = 0.439806
Epoch 243
Validation prc-auc = 0.433186
Epoch 244
Loss = 6.6628e-02, PNorm = 97.7449, GNorm = 1.0555, lr_0 = 6.6625e-04
Validation prc-auc = 0.431321
Epoch 245
Validation prc-auc = 0.414283
Epoch 246
Validation prc-auc = 0.428286
Epoch 247
Validation prc-auc = 0.419658
Epoch 248
Validation prc-auc = 0.428629
Epoch 249
Loss = 2.4759e-02, PNorm = 97.8523, GNorm = 1.0839, lr_0 = 6.6056e-04
Validation prc-auc = 0.430381
Epoch 250
Validation prc-auc = 0.432516
Epoch 251
Validation prc-auc = 0.434193
Epoch 252
Validation prc-auc = 0.434773
Epoch 253
Validation prc-auc = 0.435811
Epoch 254
Loss = 2.1636e-02, PNorm = 97.9380, GNorm = 1.0725, lr_0 = 6.5491e-04
Validation prc-auc = 0.435315
Epoch 255
Validation prc-auc = 0.434904
Epoch 256
Validation prc-auc = 0.436027
Epoch 257
Validation prc-auc = 0.436158
Epoch 258
Validation prc-auc = 0.436796
Epoch 259
Loss = 4.9105e-03, PNorm = 97.9870, GNorm = 0.0316, lr_0 = 6.4931e-04
Validation prc-auc = 0.436779
Epoch 260
Validation prc-auc = 0.436357
Epoch 261
Validation prc-auc = 0.435624
Epoch 262
Validation prc-auc = 0.450313
Epoch 263
Validation prc-auc = 0.450017
Epoch 264
Loss = 1.7833e-03, PNorm = 98.0329, GNorm = 0.1282, lr_0 = 6.4376e-04
Validation prc-auc = 0.454083
Epoch 265
Validation prc-auc = 0.453889
Epoch 266
Validation prc-auc = 0.447639
Epoch 267
Validation prc-auc = 0.446375
Epoch 268
Validation prc-auc = 0.446230
Epoch 269
Loss = 2.3305e-03, PNorm = 98.0666, GNorm = 0.0696, lr_0 = 6.3826e-04
Validation prc-auc = 0.410501
Epoch 270
Validation prc-auc = 0.406023
Epoch 271
Validation prc-auc = 0.405918
Epoch 272
Validation prc-auc = 0.405511
Epoch 273
Validation prc-auc = 0.414536
Epoch 274
Loss = 1.9820e-04, PNorm = 98.0991, GNorm = 0.0103, lr_0 = 6.3280e-04
Validation prc-auc = 0.414576
Epoch 275
Validation prc-auc = 0.417543
Epoch 276
Validation prc-auc = 0.433232
Epoch 277
Validation prc-auc = 0.432633
Epoch 278
Validation prc-auc = 0.432535
Epoch 279
Loss = 2.6805e-03, PNorm = 98.1233, GNorm = 0.0821, lr_0 = 6.2739e-04
Validation prc-auc = 0.417496
Epoch 280
Validation prc-auc = 0.416932
Epoch 281
Validation prc-auc = 0.416804
Epoch 282
Validation prc-auc = 0.421914
Epoch 283
Validation prc-auc = 0.415966
Epoch 284
Loss = 3.0147e-02, PNorm = 98.1488, GNorm = 1.0095, lr_0 = 6.2203e-04
Validation prc-auc = 0.416837
Epoch 285
Validation prc-auc = 0.432068
Epoch 286
Validation prc-auc = 0.438440
Epoch 287
Validation prc-auc = 0.446805
Epoch 288
Validation prc-auc = 0.432148
Epoch 289
Loss = 9.8165e-04, PNorm = 98.1835, GNorm = 0.0511, lr_0 = 6.1671e-04
Validation prc-auc = 0.432821
Epoch 290
Validation prc-auc = 0.433161
Epoch 291
Validation prc-auc = 0.433316
Epoch 292
Validation prc-auc = 0.433854
Epoch 293
Validation prc-auc = 0.434003
Epoch 294
Loss = 1.0291e-03, PNorm = 98.2057, GNorm = 0.0081, lr_0 = 6.1144e-04
Validation prc-auc = 0.433873
Epoch 295
Validation prc-auc = 0.433454
Epoch 296
Validation prc-auc = 0.433131
Epoch 297
Validation prc-auc = 0.433132
Epoch 298
Validation prc-auc = 0.432917
Epoch 299
Loss = 1.1475e-03, PNorm = 98.2229, GNorm = 0.0576, lr_0 = 6.0621e-04
Validation prc-auc = 0.433465
Model 0 best validation prc-auc = 0.477077 on epoch 65
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.215171
Ensemble test prc-auc = 0.215171
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.362497
Epoch 1
Validation prc-auc = 0.373475
Epoch 2
Validation prc-auc = 0.360164
Epoch 3
Validation prc-auc = 0.307196
Epoch 4
Loss = 5.7471e-01, PNorm = 87.5331, GNorm = 2.4418, lr_0 = 6.5000e-04
Validation prc-auc = 0.347058
Epoch 5
Validation prc-auc = 0.434601
Epoch 6
Validation prc-auc = 0.422351
Epoch 7
Validation prc-auc = 0.326135
Epoch 8
Validation prc-auc = 0.252511
Epoch 9
Loss = 6.0446e-01, PNorm = 87.8801, GNorm = 1.7077, lr_0 = 9.9743e-04
Validation prc-auc = 0.387881
Epoch 10
Validation prc-auc = 0.369805
Epoch 11
Validation prc-auc = 0.403868
Epoch 12
Validation prc-auc = 0.389082
Epoch 13
Validation prc-auc = 0.323228
Epoch 14
Loss = 4.4677e-01, PNorm = 88.2487, GNorm = 2.5919, lr_0 = 9.8890e-04
Validation prc-auc = 0.404478
Epoch 15
Validation prc-auc = 0.360953
Epoch 16
Validation prc-auc = 0.297174
Epoch 17
Validation prc-auc = 0.311929
Epoch 18
Validation prc-auc = 0.368593
Epoch 19
Loss = 3.9159e-01, PNorm = 88.5175, GNorm = 2.1423, lr_0 = 9.8045e-04
Validation prc-auc = 0.432622
Epoch 20
Validation prc-auc = 0.413820
Epoch 21
Validation prc-auc = 0.291417
Epoch 22
Validation prc-auc = 0.358779
Epoch 23
Validation prc-auc = 0.292472
Epoch 24
Loss = 4.4336e-01, PNorm = 88.7316, GNorm = 3.4277, lr_0 = 9.7207e-04
Validation prc-auc = 0.299142
Epoch 25
Validation prc-auc = 0.390284
Epoch 26
Validation prc-auc = 0.403603
Epoch 27
Validation prc-auc = 0.393303
Epoch 28
Validation prc-auc = 0.403191
Epoch 29
Loss = 3.1907e-01, PNorm = 88.9276, GNorm = 1.3664, lr_0 = 9.6376e-04
Validation prc-auc = 0.406497
Epoch 30
Validation prc-auc = 0.420888
Epoch 31
Validation prc-auc = 0.349849
Epoch 32
Validation prc-auc = 0.328070
Epoch 33
Validation prc-auc = 0.418059
Epoch 34
Loss = 1.6535e-01, PNorm = 89.1480, GNorm = 1.4396, lr_0 = 9.5552e-04
Validation prc-auc = 0.420485
Epoch 35
Validation prc-auc = 0.415010
Epoch 36
Validation prc-auc = 0.374750
Epoch 37
Validation prc-auc = 0.373908
Epoch 38
Validation prc-auc = 0.446249
Epoch 39
Loss = 2.0147e-01, PNorm = 89.3799, GNorm = 2.6395, lr_0 = 9.4735e-04
Validation prc-auc = 0.430924
Epoch 40
Validation prc-auc = 0.392741
Epoch 41
Validation prc-auc = 0.300909
Epoch 42
Validation prc-auc = 0.249491
Epoch 43
Validation prc-auc = 0.394501
Epoch 44
Loss = 3.1063e-01, PNorm = 89.6398, GNorm = 2.9956, lr_0 = 9.3925e-04
Validation prc-auc = 0.431657
Epoch 45
Validation prc-auc = 0.417422
Epoch 46
Validation prc-auc = 0.425411
Epoch 47
Validation prc-auc = 0.399438
Epoch 48
Validation prc-auc = 0.402946
Epoch 49
Loss = 2.7173e-01, PNorm = 89.9265, GNorm = 1.9139, lr_0 = 9.3122e-04
Validation prc-auc = 0.424633
Epoch 50
Validation prc-auc = 0.399071
Epoch 51
Validation prc-auc = 0.367465
Epoch 52
Validation prc-auc = 0.397806
Epoch 53
Validation prc-auc = 0.430022
Epoch 54
Loss = 1.8999e-01, PNorm = 90.1864, GNorm = 1.3170, lr_0 = 9.2326e-04
Validation prc-auc = 0.391149
Epoch 55
Validation prc-auc = 0.429319
Epoch 56
Validation prc-auc = 0.441753
Epoch 57
Validation prc-auc = 0.446572
Epoch 58
Validation prc-auc = 0.378147
Epoch 59
Loss = 2.1211e-01, PNorm = 90.4335, GNorm = 2.3896, lr_0 = 9.1537e-04
Validation prc-auc = 0.403177
Epoch 60
Validation prc-auc = 0.426988
Epoch 61
Validation prc-auc = 0.436805
Epoch 62
Validation prc-auc = 0.453500
Epoch 63
Validation prc-auc = 0.456654
Epoch 64
Loss = 1.2875e-01, PNorm = 90.7174, GNorm = 1.2546, lr_0 = 9.0754e-04
Validation prc-auc = 0.449807
Epoch 65
Validation prc-auc = 0.444741
Epoch 66
Validation prc-auc = 0.438521
Epoch 67
Validation prc-auc = 0.408453
Epoch 68
Validation prc-auc = 0.399117
Epoch 69
Loss = 1.0743e-01, PNorm = 90.9736, GNorm = 0.7323, lr_0 = 8.9978e-04
Validation prc-auc = 0.420213
Epoch 70
Validation prc-auc = 0.401436
Epoch 71
Validation prc-auc = 0.397209
Epoch 72
Validation prc-auc = 0.365859
Epoch 73
Validation prc-auc = 0.314413
Epoch 74
Loss = 1.5363e-01, PNorm = 91.2264, GNorm = 1.4410, lr_0 = 8.9209e-04
Validation prc-auc = 0.353852
Epoch 75
Validation prc-auc = 0.369093
Epoch 76
Validation prc-auc = 0.286124
Epoch 77
Validation prc-auc = 0.283753
Epoch 78
Validation prc-auc = 0.297360
Epoch 79
Loss = 1.0201e-01, PNorm = 91.4624, GNorm = 1.2629, lr_0 = 8.8447e-04
Validation prc-auc = 0.291513
Epoch 80
Validation prc-auc = 0.286815
Epoch 81
Validation prc-auc = 0.354887
Epoch 82
Validation prc-auc = 0.429403
Epoch 83
Validation prc-auc = 0.448080
Epoch 84
Loss = 3.5860e-02, PNorm = 91.7189, GNorm = 0.9877, lr_0 = 8.7691e-04
Validation prc-auc = 0.455732
Epoch 85
Validation prc-auc = 0.445887
Epoch 86
Validation prc-auc = 0.421709
Epoch 87
Validation prc-auc = 0.413750
Epoch 88
Validation prc-auc = 0.419208
Epoch 89
Loss = 1.4403e-01, PNorm = 92.0226, GNorm = 0.9506, lr_0 = 8.6941e-04
Validation prc-auc = 0.410183
Epoch 90
Validation prc-auc = 0.412648
Epoch 91
Validation prc-auc = 0.415619
Epoch 92
Validation prc-auc = 0.415948
Epoch 93
Validation prc-auc = 0.413295
Epoch 94
Loss = 1.4117e-01, PNorm = 92.2977, GNorm = 2.4055, lr_0 = 8.6198e-04
Validation prc-auc = 0.432744
Epoch 95
Validation prc-auc = 0.432498
Epoch 96
Validation prc-auc = 0.430308
Epoch 97
Validation prc-auc = 0.427956
Epoch 98
Validation prc-auc = 0.416071
Epoch 99
Loss = 1.0486e-01, PNorm = 92.5756, GNorm = 1.3513, lr_0 = 8.5461e-04
Validation prc-auc = 0.430817
Epoch 100
Validation prc-auc = 0.458387
Epoch 101
Validation prc-auc = 0.457987
Epoch 102
Validation prc-auc = 0.468279
Epoch 103
Validation prc-auc = 0.405552
Epoch 104
Loss = 2.9635e-01, PNorm = 92.9309, GNorm = 3.6570, lr_0 = 8.4730e-04
Validation prc-auc = 0.458990
Epoch 105
Validation prc-auc = 0.427405
Epoch 106
Validation prc-auc = 0.435064
Epoch 107
Validation prc-auc = 0.436471
Epoch 108
Validation prc-auc = 0.446655
Epoch 109
Loss = 5.6122e-02, PNorm = 93.2668, GNorm = 1.3427, lr_0 = 8.4006e-04
Validation prc-auc = 0.463776
Epoch 110
Validation prc-auc = 0.437204
Epoch 111
Validation prc-auc = 0.414979
Epoch 112
Validation prc-auc = 0.432884
Epoch 113
Validation prc-auc = 0.434954
Epoch 114
Loss = 4.0623e-02, PNorm = 93.5914, GNorm = 0.5971, lr_0 = 8.3288e-04
Validation prc-auc = 0.430121
Epoch 115
Validation prc-auc = 0.455279
Epoch 116
Validation prc-auc = 0.469207
Epoch 117
Validation prc-auc = 0.451408
Epoch 118
Validation prc-auc = 0.442795
Epoch 119
Loss = 4.4493e-02, PNorm = 93.8913, GNorm = 1.0471, lr_0 = 8.2576e-04
Validation prc-auc = 0.437362
Epoch 120
Validation prc-auc = 0.438491
Epoch 121
Validation prc-auc = 0.432409
Epoch 122
Validation prc-auc = 0.433691
Epoch 123
Validation prc-auc = 0.427856
Epoch 124
Loss = 6.0285e-02, PNorm = 94.1112, GNorm = 0.4898, lr_0 = 8.1870e-04
Validation prc-auc = 0.409716
Epoch 125
Validation prc-auc = 0.398687
Epoch 126
Validation prc-auc = 0.404561
Epoch 127
Validation prc-auc = 0.415836
Epoch 128
Validation prc-auc = 0.419848
Epoch 129
Loss = 1.0945e-02, PNorm = 94.2900, GNorm = 0.5024, lr_0 = 8.1170e-04
Validation prc-auc = 0.422735
Epoch 130
Validation prc-auc = 0.439963
Epoch 131
Validation prc-auc = 0.426693
Epoch 132
Validation prc-auc = 0.426195
Epoch 133
Validation prc-auc = 0.427817
Epoch 134
Loss = 1.4350e-02, PNorm = 94.4324, GNorm = 0.9047, lr_0 = 8.0476e-04
Validation prc-auc = 0.432892
Epoch 135
Validation prc-auc = 0.435820
Epoch 136
Validation prc-auc = 0.433113
Epoch 137
Validation prc-auc = 0.430046
Epoch 138
Validation prc-auc = 0.422264
Epoch 139
Loss = 1.0259e-02, PNorm = 94.5505, GNorm = 0.4714, lr_0 = 7.9788e-04
Validation prc-auc = 0.423306
Epoch 140
Validation prc-auc = 0.421479
Epoch 141
Validation prc-auc = 0.420942
Epoch 142
Validation prc-auc = 0.432918
Epoch 143
Validation prc-auc = 0.422745
Epoch 144
Loss = 1.6115e-02, PNorm = 94.6695, GNorm = 0.0770, lr_0 = 7.9106e-04
Validation prc-auc = 0.420235
Epoch 145
Validation prc-auc = 0.428341
Epoch 146
Validation prc-auc = 0.425984
Epoch 147
Validation prc-auc = 0.424473
Epoch 148
Validation prc-auc = 0.424060
Epoch 149
Loss = 3.3629e-02, PNorm = 94.7952, GNorm = 0.9418, lr_0 = 7.8430e-04
Validation prc-auc = 0.416380
Epoch 150
Validation prc-auc = 0.419121
Epoch 151
Validation prc-auc = 0.425806
Epoch 152
Validation prc-auc = 0.413874
Epoch 153
Validation prc-auc = 0.423797
Epoch 154
Loss = 1.4956e-02, PNorm = 94.9784, GNorm = 2.0763, lr_0 = 7.7759e-04
Validation prc-auc = 0.434036
Epoch 155
Validation prc-auc = 0.435572
Epoch 156
Validation prc-auc = 0.434682
Epoch 157
Validation prc-auc = 0.432721
Epoch 158
Validation prc-auc = 0.439676
Epoch 159
Loss = 6.6053e-02, PNorm = 95.1568, GNorm = 2.0429, lr_0 = 7.7095e-04
Validation prc-auc = 0.431471
Epoch 160
Validation prc-auc = 0.419799
Epoch 161
Validation prc-auc = 0.415386
Epoch 162
Validation prc-auc = 0.415725
Epoch 163
Validation prc-auc = 0.407419
Epoch 164
Loss = 7.6386e-02, PNorm = 95.3326, GNorm = 0.8157, lr_0 = 7.6436e-04
Validation prc-auc = 0.423183
Epoch 165
Validation prc-auc = 0.419090
Epoch 166
Validation prc-auc = 0.425196
Epoch 167
Validation prc-auc = 0.430873
Epoch 168
Validation prc-auc = 0.429507
Epoch 169
Loss = 2.3335e-02, PNorm = 95.4952, GNorm = 1.7371, lr_0 = 7.5782e-04
Validation prc-auc = 0.432665
Epoch 170
Validation prc-auc = 0.442776
Epoch 171
Validation prc-auc = 0.451910
Epoch 172
Validation prc-auc = 0.444881
Epoch 173
Validation prc-auc = 0.418125
Epoch 174
Loss = 7.6385e-02, PNorm = 95.6672, GNorm = 1.8193, lr_0 = 7.5134e-04
Validation prc-auc = 0.409125
Epoch 175
Validation prc-auc = 0.434634
Epoch 176
Validation prc-auc = 0.413389
Epoch 177
Validation prc-auc = 0.411648
Epoch 178
Validation prc-auc = 0.408666
Epoch 179
Loss = 2.8636e-02, PNorm = 95.8533, GNorm = 0.4316, lr_0 = 7.4492e-04
Validation prc-auc = 0.407018
Epoch 180
Validation prc-auc = 0.405944
Epoch 181
Validation prc-auc = 0.397092
Epoch 182
Validation prc-auc = 0.389230
Epoch 183
Validation prc-auc = 0.388730
Epoch 184
Loss = 1.3997e-02, PNorm = 96.0148, GNorm = 1.2273, lr_0 = 7.3855e-04
Validation prc-auc = 0.379806
Epoch 185
Validation prc-auc = 0.382215
Epoch 186
Validation prc-auc = 0.395183
Epoch 187
Validation prc-auc = 0.423576
Epoch 188
Validation prc-auc = 0.429422
Epoch 189
Loss = 3.1718e-02, PNorm = 96.1854, GNorm = 1.3643, lr_0 = 7.3224e-04
Validation prc-auc = 0.430683
Epoch 190
Validation prc-auc = 0.425846
Epoch 191
Validation prc-auc = 0.423840
Epoch 192
Validation prc-auc = 0.415985
Epoch 193
Validation prc-auc = 0.429555
Epoch 194
Loss = 1.5061e-02, PNorm = 96.3408, GNorm = 0.6105, lr_0 = 7.2598e-04
Validation prc-auc = 0.419225
Epoch 195
Validation prc-auc = 0.410530
Epoch 196
Validation prc-auc = 0.411891
Epoch 197
Validation prc-auc = 0.412670
Epoch 198
Validation prc-auc = 0.426483
Epoch 199
Loss = 1.6642e-02, PNorm = 96.4669, GNorm = 0.0874, lr_0 = 7.1977e-04
Validation prc-auc = 0.434059
Epoch 200
Validation prc-auc = 0.423587
Epoch 201
Validation prc-auc = 0.426260
Epoch 202
Validation prc-auc = 0.425707
Epoch 203
Validation prc-auc = 0.428406
Epoch 204
Loss = 8.2033e-03, PNorm = 96.5753, GNorm = 0.4415, lr_0 = 7.1362e-04
Validation prc-auc = 0.427029
Epoch 205
Validation prc-auc = 0.428012
Epoch 206
Validation prc-auc = 0.420726
Epoch 207
Validation prc-auc = 0.413773
Epoch 208
Validation prc-auc = 0.405794
Epoch 209
Loss = 7.1220e-02, PNorm = 96.6725, GNorm = 0.2976, lr_0 = 7.0752e-04
Validation prc-auc = 0.422869
Epoch 210
Validation prc-auc = 0.420059
Epoch 211
Validation prc-auc = 0.426741
Epoch 212
Validation prc-auc = 0.426442
Epoch 213
Validation prc-auc = 0.407242
Epoch 214
Loss = 8.1410e-02, PNorm = 96.7942, GNorm = 1.3427, lr_0 = 7.0147e-04
Validation prc-auc = 0.407128
Epoch 215
Validation prc-auc = 0.419716
Epoch 216
Validation prc-auc = 0.409848
Epoch 217
Validation prc-auc = 0.400414
Epoch 218
Validation prc-auc = 0.398776
Epoch 219
Loss = 7.0814e-02, PNorm = 96.9171, GNorm = 0.9061, lr_0 = 6.9548e-04
Validation prc-auc = 0.405125
Epoch 220
Validation prc-auc = 0.423178
Epoch 221
Validation prc-auc = 0.427428
Epoch 222
Validation prc-auc = 0.421362
Epoch 223
Validation prc-auc = 0.425184
Epoch 224
Loss = 2.3581e-03, PNorm = 97.0392, GNorm = 0.0618, lr_0 = 6.8953e-04
Validation prc-auc = 0.426541
Epoch 225
Validation prc-auc = 0.432797
Epoch 226
Validation prc-auc = 0.434541
Epoch 227
Validation prc-auc = 0.444616
Epoch 228
Validation prc-auc = 0.439158
Epoch 229
Loss = 2.1016e-03, PNorm = 97.1366, GNorm = 0.1222, lr_0 = 6.8364e-04
Validation prc-auc = 0.423143
Epoch 230
Validation prc-auc = 0.413378
Epoch 231
Validation prc-auc = 0.414064
Epoch 232
Validation prc-auc = 0.412426
Epoch 233
Validation prc-auc = 0.409638
Epoch 234
Loss = 2.6156e-02, PNorm = 97.2006, GNorm = 0.4866, lr_0 = 6.7779e-04
Validation prc-auc = 0.409020
Epoch 235
Validation prc-auc = 0.415939
Epoch 236
Validation prc-auc = 0.419089
Epoch 237
Validation prc-auc = 0.418837
Epoch 238
Validation prc-auc = 0.410577
Epoch 239
Loss = 5.1934e-03, PNorm = 97.2748, GNorm = 0.2979, lr_0 = 6.7200e-04
Validation prc-auc = 0.397048
Epoch 240
Validation prc-auc = 0.415840
Epoch 241
Validation prc-auc = 0.427551
Epoch 242
Validation prc-auc = 0.426369
Epoch 243
Validation prc-auc = 0.412687
Epoch 244
Loss = 3.5318e-02, PNorm = 97.4090, GNorm = 1.6135, lr_0 = 6.6625e-04
Validation prc-auc = 0.406118
Epoch 245
Validation prc-auc = 0.409856
Epoch 246
Validation prc-auc = 0.413975
Epoch 247
Validation prc-auc = 0.395956
Epoch 248
Validation prc-auc = 0.357268
Epoch 249
Loss = 1.0543e-02, PNorm = 97.6909, GNorm = 0.5607, lr_0 = 6.6056e-04
Validation prc-auc = 0.333036
Epoch 250
Validation prc-auc = 0.361785
Epoch 251
Validation prc-auc = 0.356519
Epoch 252
Validation prc-auc = 0.395949
Epoch 253
Validation prc-auc = 0.407393
Epoch 254
Loss = 3.7503e-02, PNorm = 97.9549, GNorm = 1.4698, lr_0 = 6.5491e-04
Validation prc-auc = 0.430491
Epoch 255
Validation prc-auc = 0.433677
Epoch 256
Validation prc-auc = 0.425420
Epoch 257
Validation prc-auc = 0.416415
Epoch 258
Validation prc-auc = 0.410698
Epoch 259
Loss = 6.3134e-02, PNorm = 98.1559, GNorm = 1.8313, lr_0 = 6.4931e-04
Validation prc-auc = 0.409104
Epoch 260
Validation prc-auc = 0.413074
Epoch 261
Validation prc-auc = 0.407982
Epoch 262
Validation prc-auc = 0.389889
Epoch 263
Validation prc-auc = 0.390445
Epoch 264
Loss = 1.4992e-02, PNorm = 98.2996, GNorm = 0.4035, lr_0 = 6.4376e-04
Validation prc-auc = 0.385515
Epoch 265
Validation prc-auc = 0.391815
Epoch 266
Validation prc-auc = 0.401407
Epoch 267
Validation prc-auc = 0.413139
Epoch 268
Validation prc-auc = 0.420079
Epoch 269
Loss = 2.4264e-02, PNorm = 98.4653, GNorm = 0.1571, lr_0 = 6.3826e-04
Validation prc-auc = 0.435628
Epoch 270
Validation prc-auc = 0.420856
Epoch 271
Validation prc-auc = 0.420383
Epoch 272
Validation prc-auc = 0.412925
Epoch 273
Validation prc-auc = 0.405403
Epoch 274
Loss = 7.6227e-04, PNorm = 98.5859, GNorm = 0.0186, lr_0 = 6.3280e-04
Validation prc-auc = 0.402540
Epoch 275
Validation prc-auc = 0.399763
Epoch 276
Validation prc-auc = 0.397767
Epoch 277
Validation prc-auc = 0.410206
Epoch 278
Validation prc-auc = 0.411266
Epoch 279
Loss = 5.5751e-03, PNorm = 98.6590, GNorm = 0.2725, lr_0 = 6.2739e-04
Validation prc-auc = 0.414863
Epoch 280
Validation prc-auc = 0.415723
Epoch 281
Validation prc-auc = 0.400970
Epoch 282
Validation prc-auc = 0.402598
Epoch 283
Validation prc-auc = 0.413584
Epoch 284
Loss = 1.4362e-02, PNorm = 98.7159, GNorm = 0.0644, lr_0 = 6.2203e-04
Validation prc-auc = 0.413286
Epoch 285
Validation prc-auc = 0.412777
Epoch 286
Validation prc-auc = 0.410730
Epoch 287
Validation prc-auc = 0.413598
Epoch 288
Validation prc-auc = 0.427388
Epoch 289
Loss = 4.7019e-03, PNorm = 98.7730, GNorm = 0.2718, lr_0 = 6.1671e-04
Validation prc-auc = 0.425908
Epoch 290
Validation prc-auc = 0.412194
Epoch 291
Validation prc-auc = 0.412296
Epoch 292
Validation prc-auc = 0.412085
Epoch 293
Validation prc-auc = 0.414612
Epoch 294
Loss = 1.2342e-02, PNorm = 98.8311, GNorm = 0.4724, lr_0 = 6.1144e-04
Validation prc-auc = 0.430093
Epoch 295
Validation prc-auc = 0.426542
Epoch 296
Validation prc-auc = 0.413056
Epoch 297
Validation prc-auc = 0.412788
Epoch 298
Validation prc-auc = 0.417391
Epoch 299
Loss = 1.4406e-04, PNorm = 98.8744, GNorm = 0.0060, lr_0 = 6.0621e-04
Validation prc-auc = 0.417506
Model 0 best validation prc-auc = 0.469207 on epoch 116
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.324470
Ensemble test prc-auc = 0.324470
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.364343
Epoch 1
Validation prc-auc = 0.378904
Epoch 2
Validation prc-auc = 0.412787
Epoch 3
Validation prc-auc = 0.383478
Epoch 4
Loss = 5.9822e-01, PNorm = 87.5359, GNorm = 2.1516, lr_0 = 6.5000e-04
Validation prc-auc = 0.389612
Epoch 5
Validation prc-auc = 0.373812
Epoch 6
Validation prc-auc = 0.286704
Epoch 7
Validation prc-auc = 0.292569
Epoch 8
Validation prc-auc = 0.361280
Epoch 9
Loss = 5.7092e-01, PNorm = 87.8888, GNorm = 3.8918, lr_0 = 9.9743e-04
Validation prc-auc = 0.388697
Epoch 10
Validation prc-auc = 0.358671
Epoch 11
Validation prc-auc = 0.222391
Epoch 12
Validation prc-auc = 0.345912
Epoch 13
Validation prc-auc = 0.398776
Epoch 14
Loss = 5.2123e-01, PNorm = 88.2616, GNorm = 2.3886, lr_0 = 9.8890e-04
Validation prc-auc = 0.326133
Epoch 15
Validation prc-auc = 0.197693
Epoch 16
Validation prc-auc = 0.265142
Epoch 17
Validation prc-auc = 0.411907
Epoch 18
Validation prc-auc = 0.336439
Epoch 19
Loss = 4.3689e-01, PNorm = 88.5536, GNorm = 2.2720, lr_0 = 9.8045e-04
Validation prc-auc = 0.329878
Epoch 20
Validation prc-auc = 0.372971
Epoch 21
Validation prc-auc = 0.389825
Epoch 22
Validation prc-auc = 0.345256
Epoch 23
Validation prc-auc = 0.444277
Epoch 24
Loss = 4.3746e-01, PNorm = 88.7911, GNorm = 1.8756, lr_0 = 9.7207e-04
Validation prc-auc = 0.430433
Epoch 25
Validation prc-auc = 0.422466
Epoch 26
Validation prc-auc = 0.383999
Epoch 27
Validation prc-auc = 0.338437
Epoch 28
Validation prc-auc = 0.253516
Epoch 29
Loss = 3.9019e-01, PNorm = 89.0043, GNorm = 1.3223, lr_0 = 9.6376e-04
Validation prc-auc = 0.352407
Epoch 30
Validation prc-auc = 0.436249
Epoch 31
Validation prc-auc = 0.424586
Epoch 32
Validation prc-auc = 0.338405
Epoch 33
Validation prc-auc = 0.307726
Epoch 34
Loss = 2.4986e-01, PNorm = 89.2237, GNorm = 1.8159, lr_0 = 9.5552e-04
Validation prc-auc = 0.369726
Epoch 35
Validation prc-auc = 0.439394
Epoch 36
Validation prc-auc = 0.427350
Epoch 37
Validation prc-auc = 0.382668
Epoch 38
Validation prc-auc = 0.391089
Epoch 39
Loss = 1.6503e-01, PNorm = 89.4467, GNorm = 1.2936, lr_0 = 9.4735e-04
Validation prc-auc = 0.417504
Epoch 40
Validation prc-auc = 0.427873
Epoch 41
Validation prc-auc = 0.383384
Epoch 42
Validation prc-auc = 0.375809
Epoch 43
Validation prc-auc = 0.345895
Epoch 44
Loss = 1.1299e-01, PNorm = 89.6710, GNorm = 0.8957, lr_0 = 9.3925e-04
Validation prc-auc = 0.392076
Epoch 45
Validation prc-auc = 0.421624
Epoch 46
Validation prc-auc = 0.429023
Epoch 47
Validation prc-auc = 0.409537
Epoch 48
Validation prc-auc = 0.443310
Epoch 49
Loss = 3.2602e-01, PNorm = 89.9191, GNorm = 2.6739, lr_0 = 9.3122e-04
Validation prc-auc = 0.429813
Epoch 50
Validation prc-auc = 0.422997
Epoch 51
Validation prc-auc = 0.400044
Epoch 52
Validation prc-auc = 0.376134
Epoch 53
Validation prc-auc = 0.434049
Epoch 54
Loss = 1.3206e-01, PNorm = 90.2204, GNorm = 1.2135, lr_0 = 9.2326e-04
Validation prc-auc = 0.431015
Epoch 55
Validation prc-auc = 0.435118
Epoch 56
Validation prc-auc = 0.442305
Epoch 57
Validation prc-auc = 0.442173
Epoch 58
Validation prc-auc = 0.447893
Epoch 59
Loss = 2.8289e-01, PNorm = 90.4895, GNorm = 3.0896, lr_0 = 9.1537e-04
Validation prc-auc = 0.426258
Epoch 60
Validation prc-auc = 0.401961
Epoch 61
Validation prc-auc = 0.393796
Epoch 62
Validation prc-auc = 0.415999
Epoch 63
Validation prc-auc = 0.421069
Epoch 64
Loss = 1.4572e-01, PNorm = 90.7182, GNorm = 1.3189, lr_0 = 9.0754e-04
Validation prc-auc = 0.428419
Epoch 65
Validation prc-auc = 0.436015
Epoch 66
Validation prc-auc = 0.436677
Epoch 67
Validation prc-auc = 0.426065
Epoch 68
Validation prc-auc = 0.412293
Epoch 69
Loss = 1.2047e-01, PNorm = 90.9178, GNorm = 1.3330, lr_0 = 8.9978e-04
Validation prc-auc = 0.409651
Epoch 70
Validation prc-auc = 0.387016
Epoch 71
Validation prc-auc = 0.257052
Epoch 72
Validation prc-auc = 0.245923
Epoch 73
Validation prc-auc = 0.411388
Epoch 74
Loss = 1.5884e-01, PNorm = 91.1677, GNorm = 1.1771, lr_0 = 8.9209e-04
Validation prc-auc = 0.425612
Epoch 75
Validation prc-auc = 0.409964
Epoch 76
Validation prc-auc = 0.410195
Epoch 77
Validation prc-auc = 0.407138
Epoch 78
Validation prc-auc = 0.417743
Epoch 79
Loss = 7.6625e-02, PNorm = 91.4548, GNorm = 0.8177, lr_0 = 8.8447e-04
Validation prc-auc = 0.406188
Epoch 80
Validation prc-auc = 0.403791
Epoch 81
Validation prc-auc = 0.406578
Epoch 82
Validation prc-auc = 0.409400
Epoch 83
Validation prc-auc = 0.406482
Epoch 84
Loss = 1.4174e-01, PNorm = 91.7478, GNorm = 1.4606, lr_0 = 8.7691e-04
Validation prc-auc = 0.402216
Epoch 85
Validation prc-auc = 0.424345
Epoch 86
Validation prc-auc = 0.452129
Epoch 87
Validation prc-auc = 0.443219
Epoch 88
Validation prc-auc = 0.475632
Epoch 89
Loss = 6.7976e-02, PNorm = 92.1156, GNorm = 0.7391, lr_0 = 8.6941e-04
Validation prc-auc = 0.462620
Epoch 90
Validation prc-auc = 0.437404
Epoch 91
Validation prc-auc = 0.426837
Epoch 92
Validation prc-auc = 0.423394
Epoch 93
Validation prc-auc = 0.441173
Epoch 94
Loss = 7.2038e-02, PNorm = 92.4938, GNorm = 0.7995, lr_0 = 8.6198e-04
Validation prc-auc = 0.441822
Epoch 95
Validation prc-auc = 0.447045
Epoch 96
Validation prc-auc = 0.445022
Epoch 97
Validation prc-auc = 0.444872
Epoch 98
Validation prc-auc = 0.464868
Epoch 99
Loss = 5.3008e-02, PNorm = 92.7976, GNorm = 0.6835, lr_0 = 8.5461e-04
Validation prc-auc = 0.464391
Epoch 100
Validation prc-auc = 0.455732
Epoch 101
Validation prc-auc = 0.444805
Epoch 102
Validation prc-auc = 0.447085
Epoch 103
Validation prc-auc = 0.439940
Epoch 104
Loss = 5.0142e-02, PNorm = 93.0281, GNorm = 2.0938, lr_0 = 8.4730e-04
Validation prc-auc = 0.434257
Epoch 105
Validation prc-auc = 0.434073
Epoch 106
Validation prc-auc = 0.431284
Epoch 107
Validation prc-auc = 0.423149
Epoch 108
Validation prc-auc = 0.432331
Epoch 109
Loss = 3.0665e-02, PNorm = 93.2336, GNorm = 0.6301, lr_0 = 8.4006e-04
Validation prc-auc = 0.440636
Epoch 110
Validation prc-auc = 0.441892
Epoch 111
Validation prc-auc = 0.447388
Epoch 112
Validation prc-auc = 0.417847
Epoch 113
Validation prc-auc = 0.393809
Epoch 114
Loss = 1.3622e-01, PNorm = 93.4152, GNorm = 1.3921, lr_0 = 8.3288e-04
Validation prc-auc = 0.407840
Epoch 115
Validation prc-auc = 0.430460
Epoch 116
Validation prc-auc = 0.425467
Epoch 117
Validation prc-auc = 0.412196
Epoch 118
Validation prc-auc = 0.436857
Epoch 119
Loss = 1.6495e-01, PNorm = 93.7128, GNorm = 0.8135, lr_0 = 8.2576e-04
Validation prc-auc = 0.434986
Epoch 120
Validation prc-auc = 0.421333
Epoch 121
Validation prc-auc = 0.417047
Epoch 122
Validation prc-auc = 0.436911
Epoch 123
Validation prc-auc = 0.421985
Epoch 124
Loss = 7.2643e-02, PNorm = 94.0062, GNorm = 0.2244, lr_0 = 8.1870e-04
Validation prc-auc = 0.437177
Epoch 125
Validation prc-auc = 0.441444
Epoch 126
Validation prc-auc = 0.428406
Epoch 127
Validation prc-auc = 0.427731
Epoch 128
Validation prc-auc = 0.417072
Epoch 129
Loss = 1.9617e-02, PNorm = 94.2429, GNorm = 0.4422, lr_0 = 8.1170e-04
Validation prc-auc = 0.431593
Epoch 130
Validation prc-auc = 0.438300
Epoch 131
Validation prc-auc = 0.436506
Epoch 132
Validation prc-auc = 0.441843
Epoch 133
Validation prc-auc = 0.439296
Epoch 134
Loss = 4.7207e-02, PNorm = 94.4489, GNorm = 0.4720, lr_0 = 8.0476e-04
Validation prc-auc = 0.430757
Epoch 135
Validation prc-auc = 0.434082
Epoch 136
Validation prc-auc = 0.421680
Epoch 137
Validation prc-auc = 0.418474
Epoch 138
Validation prc-auc = 0.414204
Epoch 139
Loss = 2.4529e-02, PNorm = 94.6332, GNorm = 0.6235, lr_0 = 7.9788e-04
Validation prc-auc = 0.417090
Epoch 140
Validation prc-auc = 0.433754
Epoch 141
Validation prc-auc = 0.444502
Epoch 142
Validation prc-auc = 0.444489
Epoch 143
Validation prc-auc = 0.445314
Epoch 144
Loss = 2.2719e-02, PNorm = 94.7751, GNorm = 0.4622, lr_0 = 7.9106e-04
Validation prc-auc = 0.423333
Epoch 145
Validation prc-auc = 0.437738
Epoch 146
Validation prc-auc = 0.435267
Epoch 147
Validation prc-auc = 0.434926
Epoch 148
Validation prc-auc = 0.435922
Epoch 149
Loss = 9.1507e-03, PNorm = 94.9298, GNorm = 0.3901, lr_0 = 7.8430e-04
Validation prc-auc = 0.436783
Epoch 150
Validation prc-auc = 0.419960
Epoch 151
Validation prc-auc = 0.421463
Epoch 152
Validation prc-auc = 0.421193
Epoch 153
Validation prc-auc = 0.424627
Epoch 154
Loss = 2.3568e-03, PNorm = 95.0602, GNorm = 0.0389, lr_0 = 7.7759e-04
Validation prc-auc = 0.445962
Epoch 155
Validation prc-auc = 0.443470
Epoch 156
Validation prc-auc = 0.437881
Epoch 157
Validation prc-auc = 0.435217
Epoch 158
Validation prc-auc = 0.449304
Epoch 159
Loss = 2.8358e-02, PNorm = 95.1535, GNorm = 0.1436, lr_0 = 7.7095e-04
Validation prc-auc = 0.459201
Epoch 160
Validation prc-auc = 0.452113
Epoch 161
Validation prc-auc = 0.448953
Epoch 162
Validation prc-auc = 0.435986
Epoch 163
Validation prc-auc = 0.426419
Epoch 164
Loss = 4.8287e-02, PNorm = 95.2649, GNorm = 1.4282, lr_0 = 7.6436e-04
Validation prc-auc = 0.429385
Epoch 165
Validation prc-auc = 0.452944
Epoch 166
Validation prc-auc = 0.450603
Epoch 167
Validation prc-auc = 0.428247
Epoch 168
Validation prc-auc = 0.436262
Epoch 169
Loss = 7.2034e-03, PNorm = 95.4330, GNorm = 0.1804, lr_0 = 7.5782e-04
Validation prc-auc = 0.407358
Epoch 170
Validation prc-auc = 0.415952
Epoch 171
Validation prc-auc = 0.440155
Epoch 172
Validation prc-auc = 0.453149
Epoch 173
Validation prc-auc = 0.450310
Epoch 174
Loss = 1.1767e-01, PNorm = 95.5824, GNorm = 0.3090, lr_0 = 7.5134e-04
Validation prc-auc = 0.468318
Epoch 175
Validation prc-auc = 0.461935
Epoch 176
Validation prc-auc = 0.465711
Epoch 177
Validation prc-auc = 0.430288
Epoch 178
Validation prc-auc = 0.421144
Epoch 179
Loss = 8.1745e-02, PNorm = 95.7540, GNorm = 1.2136, lr_0 = 7.4492e-04
Validation prc-auc = 0.418786
Epoch 180
Validation prc-auc = 0.415894
Epoch 181
Validation prc-auc = 0.455082
Epoch 182
Validation prc-auc = 0.435351
Epoch 183
Validation prc-auc = 0.435646
Epoch 184
Loss = 2.6776e-02, PNorm = 95.9082, GNorm = 0.1873, lr_0 = 7.3855e-04
Validation prc-auc = 0.422075
Epoch 185
Validation prc-auc = 0.423545
Epoch 186
Validation prc-auc = 0.425814
Epoch 187
Validation prc-auc = 0.433055
Epoch 188
Validation prc-auc = 0.434719
Epoch 189
Loss = 1.2719e-02, PNorm = 96.1028, GNorm = 0.1642, lr_0 = 7.3224e-04
Validation prc-auc = 0.428835
Epoch 190
Validation prc-auc = 0.429168
Epoch 191
Validation prc-auc = 0.434567
Epoch 192
Validation prc-auc = 0.438682
Epoch 193
Validation prc-auc = 0.423408
Epoch 194
Loss = 8.4888e-02, PNorm = 96.2557, GNorm = 0.6507, lr_0 = 7.2598e-04
Validation prc-auc = 0.427488
Epoch 195
Validation prc-auc = 0.432742
Epoch 196
Validation prc-auc = 0.438514
Epoch 197
Validation prc-auc = 0.427153
Epoch 198
Validation prc-auc = 0.416923
Epoch 199
Loss = 7.3202e-03, PNorm = 96.4167, GNorm = 0.2847, lr_0 = 7.1977e-04
Validation prc-auc = 0.425945
Epoch 200
Validation prc-auc = 0.417427
Epoch 201
Validation prc-auc = 0.415334
Epoch 202
Validation prc-auc = 0.429300
Epoch 203
Validation prc-auc = 0.447141
Epoch 204
Loss = 1.1192e-02, PNorm = 96.5622, GNorm = 0.3671, lr_0 = 7.1362e-04
Validation prc-auc = 0.449621
Epoch 205
Validation prc-auc = 0.450527
Epoch 206
Validation prc-auc = 0.423862
Epoch 207
Validation prc-auc = 0.404393
Epoch 208
Validation prc-auc = 0.421890
Epoch 209
Loss = 1.0303e-01, PNorm = 96.6977, GNorm = 0.1954, lr_0 = 7.0752e-04
Validation prc-auc = 0.425390
Epoch 210
Validation prc-auc = 0.420397
Epoch 211
Validation prc-auc = 0.412672
Epoch 212
Validation prc-auc = 0.419899
Epoch 213
Validation prc-auc = 0.425960
Epoch 214
Loss = 1.7184e-01, PNorm = 96.8530, GNorm = 1.0171, lr_0 = 7.0147e-04
Validation prc-auc = 0.446659
Epoch 215
Validation prc-auc = 0.460919
Epoch 216
Validation prc-auc = 0.469044
Epoch 217
Validation prc-auc = 0.472867
Epoch 218
Validation prc-auc = 0.473800
Epoch 219
Loss = 7.2705e-03, PNorm = 97.0843, GNorm = 0.3735, lr_0 = 6.9548e-04
Validation prc-auc = 0.474071
Epoch 220
Validation prc-auc = 0.472162
Epoch 221
Validation prc-auc = 0.471917
Epoch 222
Validation prc-auc = 0.467174
Epoch 223
Validation prc-auc = 0.463907
Epoch 224
Loss = 1.4793e-02, PNorm = 97.2839, GNorm = 0.4319, lr_0 = 6.8953e-04
Validation prc-auc = 0.456796
Epoch 225
Validation prc-auc = 0.455158
Epoch 226
Validation prc-auc = 0.452760
Epoch 227
Validation prc-auc = 0.432954
Epoch 228
Validation prc-auc = 0.453096
Epoch 229
Loss = 4.0212e-02, PNorm = 97.4074, GNorm = 1.7054, lr_0 = 6.8364e-04
Validation prc-auc = 0.447147
Epoch 230
Validation prc-auc = 0.440324
Epoch 231
Validation prc-auc = 0.424420
Epoch 232
Validation prc-auc = 0.414360
Epoch 233
Validation prc-auc = 0.412715
Epoch 234
Loss = 6.2255e-02, PNorm = 97.4886, GNorm = 2.3180, lr_0 = 6.7779e-04
Validation prc-auc = 0.417933
Epoch 235
Validation prc-auc = 0.428689
Epoch 236
Validation prc-auc = 0.431293
Epoch 237
Validation prc-auc = 0.434017
Epoch 238
Validation prc-auc = 0.434798
Epoch 239
Loss = 1.3155e-02, PNorm = 97.5749, GNorm = 0.4299, lr_0 = 6.7200e-04
Validation prc-auc = 0.436648
Epoch 240
Validation prc-auc = 0.437841
Epoch 241
Validation prc-auc = 0.439312
Epoch 242
Validation prc-auc = 0.439893
Epoch 243
Validation prc-auc = 0.440436
Epoch 244
Loss = 2.3567e-03, PNorm = 97.6410, GNorm = 0.0119, lr_0 = 6.6625e-04
Validation prc-auc = 0.439895
Epoch 245
Validation prc-auc = 0.446969
Epoch 246
Validation prc-auc = 0.447821
Epoch 247
Validation prc-auc = 0.449786
Epoch 248
Validation prc-auc = 0.452373
Epoch 249
Loss = 2.3256e-03, PNorm = 97.6864, GNorm = 0.0380, lr_0 = 6.6056e-04
Validation prc-auc = 0.451227
Epoch 250
Validation prc-auc = 0.452138
Epoch 251
Validation prc-auc = 0.445652
Epoch 252
Validation prc-auc = 0.441339
Epoch 253
Validation prc-auc = 0.442963
Epoch 254
Loss = 2.4891e-03, PNorm = 97.7295, GNorm = 0.1317, lr_0 = 6.5491e-04
Validation prc-auc = 0.443757
Epoch 255
Validation prc-auc = 0.435012
Epoch 256
Validation prc-auc = 0.429359
Epoch 257
Validation prc-auc = 0.426389
Epoch 258
Validation prc-auc = 0.437291
Epoch 259
Loss = 8.6649e-03, PNorm = 97.7934, GNorm = 0.7370, lr_0 = 6.4931e-04
Validation prc-auc = 0.434879
Epoch 260
Validation prc-auc = 0.425269
Epoch 261
Validation prc-auc = 0.423140
Epoch 262
Validation prc-auc = 0.423120
Epoch 263
Validation prc-auc = 0.422378
Epoch 264
Loss = 8.9774e-04, PNorm = 97.8725, GNorm = 0.0176, lr_0 = 6.4376e-04
Validation prc-auc = 0.421740
Epoch 265
Validation prc-auc = 0.420064
Epoch 266
Validation prc-auc = 0.419376
Epoch 267
Validation prc-auc = 0.413464
Epoch 268
Validation prc-auc = 0.409597
Epoch 269
Loss = 4.9637e-03, PNorm = 97.9259, GNorm = 0.1994, lr_0 = 6.3826e-04
Validation prc-auc = 0.409595
Epoch 270
Validation prc-auc = 0.427433
Epoch 271
Validation prc-auc = 0.436079
Epoch 272
Validation prc-auc = 0.437783
Epoch 273
Validation prc-auc = 0.431534
Epoch 274
Loss = 1.9957e-03, PNorm = 98.0005, GNorm = 0.0322, lr_0 = 6.3280e-04
Validation prc-auc = 0.434336
Epoch 275
Validation prc-auc = 0.433015
Epoch 276
Validation prc-auc = 0.438091
Epoch 277
Validation prc-auc = 0.439262
Epoch 278
Validation prc-auc = 0.438551
Epoch 279
Loss = 2.5497e-03, PNorm = 98.0992, GNorm = 0.1171, lr_0 = 6.2739e-04
Validation prc-auc = 0.430845
Epoch 280
Validation prc-auc = 0.432638
Epoch 281
Validation prc-auc = 0.431933
Epoch 282
Validation prc-auc = 0.430653
Epoch 283
Validation prc-auc = 0.430084
Epoch 284
Loss = 1.6296e-02, PNorm = 98.1956, GNorm = 0.8808, lr_0 = 6.2203e-04
Validation prc-auc = 0.431635
Epoch 285
Validation prc-auc = 0.426373
Epoch 286
Validation prc-auc = 0.427447
Epoch 287
Validation prc-auc = 0.426098
Epoch 288
Validation prc-auc = 0.436601
Epoch 289
Loss = 1.1259e-02, PNorm = 98.2849, GNorm = 0.5822, lr_0 = 6.1671e-04
Validation prc-auc = 0.440763
Epoch 290
Validation prc-auc = 0.455303
Epoch 291
Validation prc-auc = 0.454410
Epoch 292
Validation prc-auc = 0.452087
Epoch 293
Validation prc-auc = 0.453944
Epoch 294
Loss = 1.5006e-03, PNorm = 98.3577, GNorm = 0.0526, lr_0 = 6.1144e-04
Validation prc-auc = 0.452547
Epoch 295
Validation prc-auc = 0.452220
Epoch 296
Validation prc-auc = 0.453103
Epoch 297
Validation prc-auc = 0.454446
Epoch 298
Validation prc-auc = 0.454486
Epoch 299
Loss = 8.6996e-03, PNorm = 98.4146, GNorm = 0.6015, lr_0 = 6.0621e-04
Validation prc-auc = 0.455754
Model 0 best validation prc-auc = 0.475632 on epoch 88
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.322710
Ensemble test prc-auc = 0.322710
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.414257
Epoch 1
Validation prc-auc = 0.445717
Epoch 2
Validation prc-auc = 0.365733
Epoch 3
Validation prc-auc = 0.444811
Epoch 4
Loss = 4.8797e-01, PNorm = 87.5371, GNorm = 1.7777, lr_0 = 6.5000e-04
Validation prc-auc = 0.415945
Epoch 5
Validation prc-auc = 0.376035
Epoch 6
Validation prc-auc = 0.293751
Epoch 7
Validation prc-auc = 0.388780
Epoch 8
Validation prc-auc = 0.394937
Epoch 9
Loss = 5.0462e-01, PNorm = 87.8804, GNorm = 2.1611, lr_0 = 9.9743e-04
Validation prc-auc = 0.265021
Epoch 10
Validation prc-auc = 0.346012
Epoch 11
Validation prc-auc = 0.405337
Epoch 12
Validation prc-auc = 0.208550
Epoch 13
Validation prc-auc = 0.385691
Epoch 14
Loss = 4.5855e-01, PNorm = 88.2491, GNorm = 2.3881, lr_0 = 9.8890e-04
Validation prc-auc = 0.388243
Epoch 15
Validation prc-auc = 0.241614
Epoch 16
Validation prc-auc = 0.259851
Epoch 17
Validation prc-auc = 0.258495
Epoch 18
Validation prc-auc = 0.264108
Epoch 19
Loss = 4.4122e-01, PNorm = 88.5266, GNorm = 1.7841, lr_0 = 9.8045e-04
Validation prc-auc = 0.310589
Epoch 20
Validation prc-auc = 0.287303
Epoch 21
Validation prc-auc = 0.349001
Epoch 22
Validation prc-auc = 0.289425
Epoch 23
Validation prc-auc = 0.339178
Epoch 24
Loss = 3.3387e-01, PNorm = 88.7591, GNorm = 1.9345, lr_0 = 9.7207e-04
Validation prc-auc = 0.260578
Epoch 25
Validation prc-auc = 0.382775
Epoch 26
Validation prc-auc = 0.438916
Epoch 27
Validation prc-auc = 0.426842
Epoch 28
Validation prc-auc = 0.349606
Epoch 29
Loss = 2.1804e-01, PNorm = 88.9832, GNorm = 1.0786, lr_0 = 9.6376e-04
Validation prc-auc = 0.273621
Epoch 30
Validation prc-auc = 0.256470
Epoch 31
Validation prc-auc = 0.258979
Epoch 32
Validation prc-auc = 0.382383
Epoch 33
Validation prc-auc = 0.411967
Epoch 34
Loss = 2.9461e-01, PNorm = 89.2020, GNorm = 2.6531, lr_0 = 9.5552e-04
Validation prc-auc = 0.404053
Epoch 35
Validation prc-auc = 0.350505
Epoch 36
Validation prc-auc = 0.305911
Epoch 37
Validation prc-auc = 0.188381
Epoch 38
Validation prc-auc = 0.197895
Epoch 39
Loss = 5.0872e-01, PNorm = 89.4498, GNorm = 4.5179, lr_0 = 9.4735e-04
Validation prc-auc = 0.346400
Epoch 40
Validation prc-auc = 0.402768
Epoch 41
Validation prc-auc = 0.422360
Epoch 42
Validation prc-auc = 0.404637
Epoch 43
Validation prc-auc = 0.400663
Epoch 44
Loss = 2.9902e-01, PNorm = 89.7092, GNorm = 2.6516, lr_0 = 9.3925e-04
Validation prc-auc = 0.392673
Epoch 45
Validation prc-auc = 0.404133
Epoch 46
Validation prc-auc = 0.412109
Epoch 47
Validation prc-auc = 0.432586
Epoch 48
Validation prc-auc = 0.443726
Epoch 49
Loss = 1.2571e-01, PNorm = 89.9318, GNorm = 1.4227, lr_0 = 9.3122e-04
Validation prc-auc = 0.441303
Epoch 50
Validation prc-auc = 0.450615
Epoch 51
Validation prc-auc = 0.456435
Epoch 52
Validation prc-auc = 0.434791
Epoch 53
Validation prc-auc = 0.445971
Epoch 54
Loss = 2.1089e-01, PNorm = 90.1620, GNorm = 1.5230, lr_0 = 9.2326e-04
Validation prc-auc = 0.393625
Epoch 55
Validation prc-auc = 0.422821
Epoch 56
Validation prc-auc = 0.439857
Epoch 57
Validation prc-auc = 0.433587
Epoch 58
Validation prc-auc = 0.432385
Epoch 59
Loss = 8.5892e-02, PNorm = 90.3975, GNorm = 0.4755, lr_0 = 9.1537e-04
Validation prc-auc = 0.410333
Epoch 60
Validation prc-auc = 0.335344
Epoch 61
Validation prc-auc = 0.405496
Epoch 62
Validation prc-auc = 0.425574
Epoch 63
Validation prc-auc = 0.414416
Epoch 64
Loss = 2.3434e-01, PNorm = 90.6039, GNorm = 1.3754, lr_0 = 9.0754e-04
Validation prc-auc = 0.394195
Epoch 65
Validation prc-auc = 0.348306
Epoch 66
Validation prc-auc = 0.393402
Epoch 67
Validation prc-auc = 0.406474
Epoch 68
Validation prc-auc = 0.385690
Epoch 69
Loss = 1.7540e-01, PNorm = 90.8132, GNorm = 1.0783, lr_0 = 8.9978e-04
Validation prc-auc = 0.376255
Epoch 70
Validation prc-auc = 0.405218
Epoch 71
Validation prc-auc = 0.391929
Epoch 72
Validation prc-auc = 0.385104
Epoch 73
Validation prc-auc = 0.393708
Epoch 74
Loss = 8.2244e-02, PNorm = 91.0321, GNorm = 0.6057, lr_0 = 8.9209e-04
Validation prc-auc = 0.385630
Epoch 75
Validation prc-auc = 0.386585
Epoch 76
Validation prc-auc = 0.389635
Epoch 77
Validation prc-auc = 0.410354
Epoch 78
Validation prc-auc = 0.426549
Epoch 79
Loss = 4.1745e-02, PNorm = 91.2559, GNorm = 1.1604, lr_0 = 8.8447e-04
Validation prc-auc = 0.454005
Epoch 80
Validation prc-auc = 0.452619
Epoch 81
Validation prc-auc = 0.435434
Epoch 82
Validation prc-auc = 0.403855
Epoch 83
Validation prc-auc = 0.402457
Epoch 84
Loss = 5.8828e-02, PNorm = 91.4934, GNorm = 1.0326, lr_0 = 8.7691e-04
Validation prc-auc = 0.414378
Epoch 85
Validation prc-auc = 0.409235
Epoch 86
Validation prc-auc = 0.418395
Epoch 87
Validation prc-auc = 0.418624
Epoch 88
Validation prc-auc = 0.416401
Epoch 89
Loss = 4.8727e-02, PNorm = 91.7778, GNorm = 1.0963, lr_0 = 8.6941e-04
Validation prc-auc = 0.407826
Epoch 90
Validation prc-auc = 0.404779
Epoch 91
Validation prc-auc = 0.416357
Epoch 92
Validation prc-auc = 0.423872
Epoch 93
Validation prc-auc = 0.419435
Epoch 94
Loss = 1.6906e-01, PNorm = 92.0812, GNorm = 2.0033, lr_0 = 8.6198e-04
Validation prc-auc = 0.425091
Epoch 95
Validation prc-auc = 0.439918
Epoch 96
Validation prc-auc = 0.405391
Epoch 97
Validation prc-auc = 0.441059
Epoch 98
Validation prc-auc = 0.432471
Epoch 99
Loss = 4.3102e-02, PNorm = 92.4628, GNorm = 0.6395, lr_0 = 8.5461e-04
Validation prc-auc = 0.435658
Epoch 100
Validation prc-auc = 0.432307
Epoch 101
Validation prc-auc = 0.436096
Epoch 102
Validation prc-auc = 0.451639
Epoch 103
Validation prc-auc = 0.458467
Epoch 104
Loss = 5.6969e-02, PNorm = 92.7764, GNorm = 0.8174, lr_0 = 8.4730e-04
Validation prc-auc = 0.457060
Epoch 105
Validation prc-auc = 0.454909
Epoch 106
Validation prc-auc = 0.451787
Epoch 107
Validation prc-auc = 0.428969
Epoch 108
Validation prc-auc = 0.422492
Epoch 109
Loss = 2.6618e-02, PNorm = 93.0079, GNorm = 0.5481, lr_0 = 8.4006e-04
Validation prc-auc = 0.420753
Epoch 110
Validation prc-auc = 0.422854
Epoch 111
Validation prc-auc = 0.433335
Epoch 112
Validation prc-auc = 0.443964
Epoch 113
Validation prc-auc = 0.457799
Epoch 114
Loss = 8.9891e-02, PNorm = 93.2017, GNorm = 1.9402, lr_0 = 8.3288e-04
Validation prc-auc = 0.462663
Epoch 115
Validation prc-auc = 0.451218
Epoch 116
Validation prc-auc = 0.447848
Epoch 117
Validation prc-auc = 0.447253
Epoch 118
Validation prc-auc = 0.427823
Epoch 119
Loss = 6.0726e-02, PNorm = 93.3719, GNorm = 0.6210, lr_0 = 8.2576e-04
Validation prc-auc = 0.437155
Epoch 120
Validation prc-auc = 0.423293
Epoch 121
Validation prc-auc = 0.413632
Epoch 122
Validation prc-auc = 0.410755
Epoch 123
Validation prc-auc = 0.394333
Epoch 124
Loss = 1.6890e-02, PNorm = 93.5330, GNorm = 0.4042, lr_0 = 8.1870e-04
Validation prc-auc = 0.393587
Epoch 125
Validation prc-auc = 0.400785
Epoch 126
Validation prc-auc = 0.413605
Epoch 127
Validation prc-auc = 0.418731
Epoch 128
Validation prc-auc = 0.412185
Epoch 129
Loss = 1.2244e-01, PNorm = 93.6763, GNorm = 2.2393, lr_0 = 8.1170e-04
Validation prc-auc = 0.422390
Epoch 130
Validation prc-auc = 0.435808
Epoch 131
Validation prc-auc = 0.426532
Epoch 132
Validation prc-auc = 0.424414
Epoch 133
Validation prc-auc = 0.439932
Epoch 134
Loss = 1.7200e-01, PNorm = 93.9275, GNorm = 1.3986, lr_0 = 8.0476e-04
Validation prc-auc = 0.437582
Epoch 135
Validation prc-auc = 0.437651
Epoch 136
Validation prc-auc = 0.457772
Epoch 137
Validation prc-auc = 0.435221
Epoch 138
Validation prc-auc = 0.414299
Epoch 139
Loss = 4.2237e-02, PNorm = 94.2069, GNorm = 0.9331, lr_0 = 7.9788e-04
Validation prc-auc = 0.378607
Epoch 140
Validation prc-auc = 0.378511
Epoch 141
Validation prc-auc = 0.401658
Epoch 142
Validation prc-auc = 0.404380
Epoch 143
Validation prc-auc = 0.402021
Epoch 144
Loss = 3.5636e-02, PNorm = 94.4236, GNorm = 0.2590, lr_0 = 7.9106e-04
Validation prc-auc = 0.405300
Epoch 145
Validation prc-auc = 0.403961
Epoch 146
Validation prc-auc = 0.380773
Epoch 147
Validation prc-auc = 0.377282
Epoch 148
Validation prc-auc = 0.376820
Epoch 149
Loss = 6.7442e-02, PNorm = 94.5819, GNorm = 0.7350, lr_0 = 7.8430e-04
Validation prc-auc = 0.382959
Epoch 150
Validation prc-auc = 0.355257
Epoch 151
Validation prc-auc = 0.345168
Epoch 152
Validation prc-auc = 0.383467
Epoch 153
Validation prc-auc = 0.404596
Epoch 154
Loss = 6.5364e-03, PNorm = 94.7078, GNorm = 0.2689, lr_0 = 7.7759e-04
Validation prc-auc = 0.404276
Epoch 155
Validation prc-auc = 0.400296
Epoch 156
Validation prc-auc = 0.432353
Epoch 157
Validation prc-auc = 0.433089
Epoch 158
Validation prc-auc = 0.430970
Epoch 159
Loss = 2.5449e-03, PNorm = 94.8150, GNorm = 0.0542, lr_0 = 7.7095e-04
Validation prc-auc = 0.430749
Epoch 160
Validation prc-auc = 0.413483
Epoch 161
Validation prc-auc = 0.406398
Epoch 162
Validation prc-auc = 0.404947
Epoch 163
Validation prc-auc = 0.404685
Epoch 164
Loss = 4.0134e-02, PNorm = 94.8981, GNorm = 0.5008, lr_0 = 7.6436e-04
Validation prc-auc = 0.398505
Epoch 165
Validation prc-auc = 0.407302
Epoch 166
Validation prc-auc = 0.429806
Epoch 167
Validation prc-auc = 0.431008
Epoch 168
Validation prc-auc = 0.429811
Epoch 169
Loss = 9.5468e-03, PNorm = 94.9799, GNorm = 0.3971, lr_0 = 7.5782e-04
Validation prc-auc = 0.401358
Epoch 170
Validation prc-auc = 0.407371
Epoch 171
Validation prc-auc = 0.399385
Epoch 172
Validation prc-auc = 0.414988
Epoch 173
Validation prc-auc = 0.409447
Epoch 174
Loss = 2.3796e-02, PNorm = 95.0706, GNorm = 0.3431, lr_0 = 7.5134e-04
Validation prc-auc = 0.399696
Epoch 175
Validation prc-auc = 0.387726
Epoch 176
Validation prc-auc = 0.393806
Epoch 177
Validation prc-auc = 0.403118
Epoch 178
Validation prc-auc = 0.410627
Epoch 179
Loss = 5.7222e-03, PNorm = 95.2281, GNorm = 0.0436, lr_0 = 7.4492e-04
Validation prc-auc = 0.411865
Epoch 180
Validation prc-auc = 0.401198
Epoch 181
Validation prc-auc = 0.398793
Epoch 182
Validation prc-auc = 0.398718
Epoch 183
Validation prc-auc = 0.395788
Epoch 184
Loss = 2.3215e-02, PNorm = 95.3542, GNorm = 0.3435, lr_0 = 7.3855e-04
Validation prc-auc = 0.407083
Epoch 185
Validation prc-auc = 0.399745
Epoch 186
Validation prc-auc = 0.390388
Epoch 187
Validation prc-auc = 0.398997
Epoch 188
Validation prc-auc = 0.430552
Epoch 189
Loss = 1.8562e-02, PNorm = 95.4700, GNorm = 0.7801, lr_0 = 7.3224e-04
Validation prc-auc = 0.435468
Epoch 190
Validation prc-auc = 0.440255
Epoch 191
Validation prc-auc = 0.443531
Epoch 192
Validation prc-auc = 0.443903
Epoch 193
Validation prc-auc = 0.445382
Epoch 194
Loss = 5.4535e-03, PNorm = 95.6140, GNorm = 0.1602, lr_0 = 7.2598e-04
Validation prc-auc = 0.436346
Epoch 195
Validation prc-auc = 0.440957
Epoch 196
Validation prc-auc = 0.444687
Epoch 197
Validation prc-auc = 0.445158
Epoch 198
Validation prc-auc = 0.449062
Epoch 199
Loss = 1.1869e-02, PNorm = 95.7354, GNorm = 0.1924, lr_0 = 7.1977e-04
Validation prc-auc = 0.450578
Epoch 200
Validation prc-auc = 0.446832
Epoch 201
Validation prc-auc = 0.434255
Epoch 202
Validation prc-auc = 0.432095
Epoch 203
Validation prc-auc = 0.435956
Epoch 204
Loss = 2.1571e-03, PNorm = 95.8204, GNorm = 0.0304, lr_0 = 7.1362e-04
Validation prc-auc = 0.434544
Epoch 205
Validation prc-auc = 0.424207
Epoch 206
Validation prc-auc = 0.406375
Epoch 207
Validation prc-auc = 0.406171
Epoch 208
Validation prc-auc = 0.400958
Epoch 209
Loss = 3.0162e-02, PNorm = 95.9020, GNorm = 0.9443, lr_0 = 7.0752e-04
Validation prc-auc = 0.400939
Epoch 210
Validation prc-auc = 0.401063
Epoch 211
Validation prc-auc = 0.408019
Epoch 212
Validation prc-auc = 0.414263
Epoch 213
Validation prc-auc = 0.429694
Epoch 214
Loss = 2.7737e-02, PNorm = 95.9889, GNorm = 0.9150, lr_0 = 7.0147e-04
Validation prc-auc = 0.438238
Epoch 215
Validation prc-auc = 0.437805
Epoch 216
Validation prc-auc = 0.438085
Epoch 217
Validation prc-auc = 0.445069
Epoch 218
Validation prc-auc = 0.445209
Epoch 219
Loss = 1.9584e-03, PNorm = 96.0583, GNorm = 0.0302, lr_0 = 6.9548e-04
Validation prc-auc = 0.443509
Epoch 220
Validation prc-auc = 0.442929
Epoch 221
Validation prc-auc = 0.442727
Epoch 222
Validation prc-auc = 0.442506
Epoch 223
Validation prc-auc = 0.434011
Epoch 224
Loss = 3.1194e-03, PNorm = 96.1044, GNorm = 0.0677, lr_0 = 6.8953e-04
Validation prc-auc = 0.433102
Epoch 225
Validation prc-auc = 0.423272
Epoch 226
Validation prc-auc = 0.421714
Epoch 227
Validation prc-auc = 0.421255
Epoch 228
Validation prc-auc = 0.425368
Epoch 229
Loss = 4.8832e-03, PNorm = 96.1460, GNorm = 0.0305, lr_0 = 6.8364e-04
Validation prc-auc = 0.427095
Epoch 230
Validation prc-auc = 0.428306
Epoch 231
Validation prc-auc = 0.440803
Epoch 232
Validation prc-auc = 0.422973
Epoch 233
Validation prc-auc = 0.423579
Epoch 234
Loss = 3.2502e-03, PNorm = 96.2037, GNorm = 0.1413, lr_0 = 6.7779e-04
Validation prc-auc = 0.423200
Epoch 235
Validation prc-auc = 0.423298
Epoch 236
Validation prc-auc = 0.422564
Epoch 237
Validation prc-auc = 0.422883
Epoch 238
Validation prc-auc = 0.420315
Epoch 239
Loss = 9.9433e-04, PNorm = 96.2592, GNorm = 0.0155, lr_0 = 6.7200e-04
Validation prc-auc = 0.419352
Epoch 240
Validation prc-auc = 0.419006
Epoch 241
Validation prc-auc = 0.423517
Epoch 242
Validation prc-auc = 0.423190
Epoch 243
Validation prc-auc = 0.419003
Epoch 244
Loss = 8.9944e-04, PNorm = 96.3124, GNorm = 0.0213, lr_0 = 6.6625e-04
Validation prc-auc = 0.418310
Epoch 245
Validation prc-auc = 0.418478
Epoch 246
Validation prc-auc = 0.417572
Epoch 247
Validation prc-auc = 0.422431
Epoch 248
Validation prc-auc = 0.425685
Epoch 249
Loss = 2.0141e-03, PNorm = 96.3502, GNorm = 0.1499, lr_0 = 6.6056e-04
Validation prc-auc = 0.426606
Epoch 250
Validation prc-auc = 0.425727
Epoch 251
Validation prc-auc = 0.425725
Epoch 252
Validation prc-auc = 0.426017
Epoch 253
Validation prc-auc = 0.405535
Epoch 254
Loss = 4.0765e-03, PNorm = 96.3814, GNorm = 0.3493, lr_0 = 6.5491e-04
Validation prc-auc = 0.415608
Epoch 255
Validation prc-auc = 0.414300
Epoch 256
Validation prc-auc = 0.409818
Epoch 257
Validation prc-auc = 0.409017
Epoch 258
Validation prc-auc = 0.405292
Epoch 259
Loss = 3.6600e-04, PNorm = 96.4118, GNorm = 0.0160, lr_0 = 6.4931e-04
Validation prc-auc = 0.403940
Epoch 260
Validation prc-auc = 0.396546
Epoch 261
Validation prc-auc = 0.391276
Epoch 262
Validation prc-auc = 0.390176
Epoch 263
Validation prc-auc = 0.389797
Epoch 264
Loss = 1.4159e-02, PNorm = 96.4439, GNorm = 0.1830, lr_0 = 6.4376e-04
Validation prc-auc = 0.395589
Epoch 265
Validation prc-auc = 0.396615
Epoch 266
Validation prc-auc = 0.402957
Epoch 267
Validation prc-auc = 0.419041
Epoch 268
Validation prc-auc = 0.420158
Epoch 269
Loss = 6.3007e-03, PNorm = 96.4756, GNorm = 0.0123, lr_0 = 6.3826e-04
Validation prc-auc = 0.414317
Epoch 270
Validation prc-auc = 0.414309
Epoch 271
Validation prc-auc = 0.415426
Epoch 272
Validation prc-auc = 0.415931
Epoch 273
Validation prc-auc = 0.415891
Epoch 274
Loss = 5.3513e-03, PNorm = 96.4980, GNorm = 0.0230, lr_0 = 6.3280e-04
Validation prc-auc = 0.409453
Epoch 275
Validation prc-auc = 0.409396
Epoch 276
Validation prc-auc = 0.414325
Epoch 277
Validation prc-auc = 0.421012
Epoch 278
Validation prc-auc = 0.420915
Epoch 279
Loss = 2.3230e-03, PNorm = 96.5350, GNorm = 0.2273, lr_0 = 6.2739e-04
Validation prc-auc = 0.415754
Epoch 280
Validation prc-auc = 0.415868
Epoch 281
Validation prc-auc = 0.415098
Epoch 282
Validation prc-auc = 0.414594
Epoch 283
Validation prc-auc = 0.414516
Epoch 284
Loss = 1.6948e-03, PNorm = 96.5617, GNorm = 0.1072, lr_0 = 6.2203e-04
Validation prc-auc = 0.414861
Epoch 285
Validation prc-auc = 0.415299
Epoch 286
Validation prc-auc = 0.415756
Epoch 287
Validation prc-auc = 0.415569
Epoch 288
Validation prc-auc = 0.410335
Epoch 289
Loss = 7.5381e-04, PNorm = 96.5756, GNorm = 0.0128, lr_0 = 6.1671e-04
Validation prc-auc = 0.406241
Epoch 290
Validation prc-auc = 0.405797
Epoch 291
Validation prc-auc = 0.405034
Epoch 292
Validation prc-auc = 0.404969
Epoch 293
Validation prc-auc = 0.405355
Epoch 294
Loss = 1.8623e-04, PNorm = 96.5867, GNorm = 0.0132, lr_0 = 6.1144e-04
Validation prc-auc = 0.406410
Epoch 295
Validation prc-auc = 0.406572
Epoch 296
Validation prc-auc = 0.406753
Epoch 297
Validation prc-auc = 0.406709
Epoch 298
Validation prc-auc = 0.406709
Epoch 299
Loss = 1.7388e-03, PNorm = 96.5949, GNorm = 0.4004, lr_0 = 6.0621e-04
Validation prc-auc = 0.410858
Model 0 best validation prc-auc = 0.462663 on epoch 114
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.254939
Ensemble test prc-auc = 0.254939
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.345296
Epoch 1
Validation prc-auc = 0.397866
Epoch 2
Validation prc-auc = 0.430034
Epoch 3
Validation prc-auc = 0.440411
Epoch 4
Loss = 5.4731e-01, PNorm = 87.5390, GNorm = 3.5347, lr_0 = 6.5000e-04
Validation prc-auc = 0.459056
Epoch 5
Validation prc-auc = 0.440142
Epoch 6
Validation prc-auc = 0.410282
Epoch 7
Validation prc-auc = 0.389161
Epoch 8
Validation prc-auc = 0.416211
Epoch 9
Loss = 4.8582e-01, PNorm = 87.8876, GNorm = 2.5691, lr_0 = 9.9743e-04
Validation prc-auc = 0.397884
Epoch 10
Validation prc-auc = 0.250400
Epoch 11
Validation prc-auc = 0.405982
Epoch 12
Validation prc-auc = 0.459659
Epoch 13
Validation prc-auc = 0.408564
Epoch 14
Loss = 5.4467e-01, PNorm = 88.2601, GNorm = 3.6804, lr_0 = 9.8890e-04
Validation prc-auc = 0.295985
Epoch 15
Validation prc-auc = 0.306699
Epoch 16
Validation prc-auc = 0.367702
Epoch 17
Validation prc-auc = 0.446856
Epoch 18
Validation prc-auc = 0.449918
Epoch 19
Loss = 4.2763e-01, PNorm = 88.5329, GNorm = 2.0105, lr_0 = 9.8045e-04
Validation prc-auc = 0.165246
Epoch 20
Validation prc-auc = 0.160480
Epoch 21
Validation prc-auc = 0.233128
Epoch 22
Validation prc-auc = 0.332978
Epoch 23
Validation prc-auc = 0.408284
Epoch 24
Loss = 2.8138e-01, PNorm = 88.7541, GNorm = 1.2172, lr_0 = 9.7207e-04
Validation prc-auc = 0.422616
Epoch 25
Validation prc-auc = 0.427895
Epoch 26
Validation prc-auc = 0.373255
Epoch 27
Validation prc-auc = 0.374785
Epoch 28
Validation prc-auc = 0.380712
Epoch 29
Loss = 2.8360e-01, PNorm = 88.9370, GNorm = 1.7021, lr_0 = 9.6376e-04
Validation prc-auc = 0.416409
Epoch 30
Validation prc-auc = 0.403885
Epoch 31
Validation prc-auc = 0.400126
Epoch 32
Validation prc-auc = 0.384913
Epoch 33
Validation prc-auc = 0.329666
Epoch 34
Loss = 2.3266e-01, PNorm = 89.0952, GNorm = 0.7611, lr_0 = 9.5552e-04
Validation prc-auc = 0.367957
Epoch 35
Validation prc-auc = 0.391270
Epoch 36
Validation prc-auc = 0.423032
Epoch 37
Validation prc-auc = 0.438336
Epoch 38
Validation prc-auc = 0.437280
Epoch 39
Loss = 2.8941e-01, PNorm = 89.2755, GNorm = 1.3821, lr_0 = 9.4735e-04
Validation prc-auc = 0.415361
Epoch 40
Validation prc-auc = 0.435077
Epoch 41
Validation prc-auc = 0.448832
Epoch 42
Validation prc-auc = 0.435666
Epoch 43
Validation prc-auc = 0.437539
Epoch 44
Loss = 2.3382e-01, PNorm = 89.4676, GNorm = 1.3855, lr_0 = 9.3925e-04
Validation prc-auc = 0.405176
Epoch 45
Validation prc-auc = 0.373398
Epoch 46
Validation prc-auc = 0.414066
Epoch 47
Validation prc-auc = 0.416673
Epoch 48
Validation prc-auc = 0.420503
Epoch 49
Loss = 2.0029e-01, PNorm = 89.6968, GNorm = 1.3153, lr_0 = 9.3122e-04
Validation prc-auc = 0.415284
Epoch 50
Validation prc-auc = 0.390709
Epoch 51
Validation prc-auc = 0.372362
Epoch 52
Validation prc-auc = 0.391298
Epoch 53
Validation prc-auc = 0.415039
Epoch 54
Loss = 1.6242e-01, PNorm = 89.9295, GNorm = 0.7890, lr_0 = 9.2326e-04
Validation prc-auc = 0.438070
Epoch 55
Validation prc-auc = 0.411531
Epoch 56
Validation prc-auc = 0.399209
Epoch 57
Validation prc-auc = 0.407973
Epoch 58
Validation prc-auc = 0.395717
Epoch 59
Loss = 1.4036e-01, PNorm = 90.1893, GNorm = 1.6427, lr_0 = 9.1537e-04
Validation prc-auc = 0.400304
Epoch 60
Validation prc-auc = 0.395419
Epoch 61
Validation prc-auc = 0.388922
Epoch 62
Validation prc-auc = 0.419503
Epoch 63
Validation prc-auc = 0.435858
Epoch 64
Loss = 1.8957e-01, PNorm = 90.4667, GNorm = 2.7218, lr_0 = 9.0754e-04
Validation prc-auc = 0.464657
Epoch 65
Validation prc-auc = 0.413071
Epoch 66
Validation prc-auc = 0.370161
Epoch 67
Validation prc-auc = 0.382529
Epoch 68
Validation prc-auc = 0.359286
Epoch 69
Loss = 1.7864e-01, PNorm = 90.7527, GNorm = 1.5311, lr_0 = 8.9978e-04
Validation prc-auc = 0.384062
Epoch 70
Validation prc-auc = 0.385009
Epoch 71
Validation prc-auc = 0.392312
Epoch 72
Validation prc-auc = 0.422338
Epoch 73
Validation prc-auc = 0.428115
Epoch 74
Loss = 1.3889e-01, PNorm = 91.0524, GNorm = 0.8156, lr_0 = 8.9209e-04
Validation prc-auc = 0.451996
Epoch 75
Validation prc-auc = 0.455649
Epoch 76
Validation prc-auc = 0.429359
Epoch 77
Validation prc-auc = 0.422185
Epoch 78
Validation prc-auc = 0.412395
Epoch 79
Loss = 7.0322e-02, PNorm = 91.3413, GNorm = 0.8004, lr_0 = 8.8447e-04
Validation prc-auc = 0.390330
Epoch 80
Validation prc-auc = 0.389824
Epoch 81
Validation prc-auc = 0.411408
Epoch 82
Validation prc-auc = 0.416993
Epoch 83
Validation prc-auc = 0.439918
Epoch 84
Loss = 3.1122e-02, PNorm = 91.5810, GNorm = 0.2236, lr_0 = 8.7691e-04
Validation prc-auc = 0.452628
Epoch 85
Validation prc-auc = 0.437819
Epoch 86
Validation prc-auc = 0.436546
Epoch 87
Validation prc-auc = 0.439730
Epoch 88
Validation prc-auc = 0.430953
Epoch 89
Loss = 1.0801e-01, PNorm = 91.8071, GNorm = 1.6115, lr_0 = 8.6941e-04
Validation prc-auc = 0.426735
Epoch 90
Validation prc-auc = 0.418084
Epoch 91
Validation prc-auc = 0.405089
Epoch 92
Validation prc-auc = 0.402622
Epoch 93
Validation prc-auc = 0.440571
Epoch 94
Loss = 5.1939e-02, PNorm = 92.1068, GNorm = 0.2029, lr_0 = 8.6198e-04
Validation prc-auc = 0.449639
Epoch 95
Validation prc-auc = 0.458311
Epoch 96
Validation prc-auc = 0.457185
Epoch 97
Validation prc-auc = 0.447979
Epoch 98
Validation prc-auc = 0.437788
Epoch 99
Loss = 5.1899e-02, PNorm = 92.4472, GNorm = 1.0080, lr_0 = 8.5461e-04
Validation prc-auc = 0.439931
Epoch 100
Validation prc-auc = 0.422594
Epoch 101
Validation prc-auc = 0.413981
Epoch 102
Validation prc-auc = 0.428921
Epoch 103
Validation prc-auc = 0.440874
Epoch 104
Loss = 8.9104e-02, PNorm = 92.7640, GNorm = 2.9867, lr_0 = 8.4730e-04
Validation prc-auc = 0.428423
Epoch 105
Validation prc-auc = 0.409748
Epoch 106
Validation prc-auc = 0.381940
Epoch 107
Validation prc-auc = 0.370507
Epoch 108
Validation prc-auc = 0.378424
Epoch 109
Loss = 5.0600e-02, PNorm = 93.1224, GNorm = 1.2186, lr_0 = 8.4006e-04
Validation prc-auc = 0.365994
Epoch 110
Validation prc-auc = 0.376609
Epoch 111
Validation prc-auc = 0.383729
Epoch 112
Validation prc-auc = 0.386493
Epoch 113
Validation prc-auc = 0.407749
Epoch 114
Loss = 1.8597e-02, PNorm = 93.4137, GNorm = 0.5196, lr_0 = 8.3288e-04
Validation prc-auc = 0.431694
Epoch 115
Validation prc-auc = 0.430316
Epoch 116
Validation prc-auc = 0.424006
Epoch 117
Validation prc-auc = 0.424394
Epoch 118
Validation prc-auc = 0.422010
Epoch 119
Loss = 2.6722e-02, PNorm = 93.6530, GNorm = 0.3173, lr_0 = 8.2576e-04
Validation prc-auc = 0.418397
Epoch 120
Validation prc-auc = 0.422196
Epoch 121
Validation prc-auc = 0.417587
Epoch 122
Validation prc-auc = 0.417369
Epoch 123
Validation prc-auc = 0.424537
Epoch 124
Loss = 2.0487e-01, PNorm = 93.9260, GNorm = 1.7239, lr_0 = 8.1870e-04
Validation prc-auc = 0.404541
Epoch 125
Validation prc-auc = 0.414302
Epoch 126
Validation prc-auc = 0.428606
Epoch 127
Validation prc-auc = 0.448424
Epoch 128
Validation prc-auc = 0.449362
Epoch 129
Loss = 8.8046e-02, PNorm = 94.1664, GNorm = 0.6867, lr_0 = 8.1170e-04
Validation prc-auc = 0.425553
Epoch 130
Validation prc-auc = 0.439152
Epoch 131
Validation prc-auc = 0.435322
Epoch 132
Validation prc-auc = 0.449750
Epoch 133
Validation prc-auc = 0.448728
Epoch 134
Loss = 1.0376e-02, PNorm = 94.3871, GNorm = 0.0675, lr_0 = 8.0476e-04
Validation prc-auc = 0.447994
Epoch 135
Validation prc-auc = 0.448222
Epoch 136
Validation prc-auc = 0.438339
Epoch 137
Validation prc-auc = 0.434576
Epoch 138
Validation prc-auc = 0.432094
Epoch 139
Loss = 1.1043e-01, PNorm = 94.5583, GNorm = 1.9750, lr_0 = 7.9788e-04
Validation prc-auc = 0.424299
Epoch 140
Validation prc-auc = 0.429928
Epoch 141
Validation prc-auc = 0.415639
Epoch 142
Validation prc-auc = 0.403072
Epoch 143
Validation prc-auc = 0.419170
Epoch 144
Loss = 4.0316e-02, PNorm = 94.7224, GNorm = 0.3954, lr_0 = 7.9106e-04
Validation prc-auc = 0.411471
Epoch 145
Validation prc-auc = 0.408943
Epoch 146
Validation prc-auc = 0.419275
Epoch 147
Validation prc-auc = 0.403133
Epoch 148
Validation prc-auc = 0.432943
Epoch 149
Loss = 2.5498e-02, PNorm = 94.9019, GNorm = 0.8764, lr_0 = 7.8430e-04
Validation prc-auc = 0.435508
Epoch 150
Validation prc-auc = 0.429792
Epoch 151
Validation prc-auc = 0.430070
Epoch 152
Validation prc-auc = 0.423944
Epoch 153
Validation prc-auc = 0.409907
Epoch 154
Loss = 2.9378e-02, PNorm = 95.1374, GNorm = 1.2241, lr_0 = 7.7759e-04
Validation prc-auc = 0.393908
Epoch 155
Validation prc-auc = 0.390829
Epoch 156
Validation prc-auc = 0.361915
Epoch 157
Validation prc-auc = 0.370826
Epoch 158
Validation prc-auc = 0.376311
Epoch 159
Loss = 5.9957e-02, PNorm = 95.3774, GNorm = 1.3736, lr_0 = 7.7095e-04
Validation prc-auc = 0.398176
Epoch 160
Validation prc-auc = 0.402802
Epoch 161
Validation prc-auc = 0.428348
Epoch 162
Validation prc-auc = 0.416310
Epoch 163
Validation prc-auc = 0.416049
Epoch 164
Loss = 8.5327e-02, PNorm = 95.6044, GNorm = 0.8951, lr_0 = 7.6436e-04
Validation prc-auc = 0.429178
Epoch 165
Validation prc-auc = 0.433753
Epoch 166
Validation prc-auc = 0.437603
Epoch 167
Validation prc-auc = 0.440721
Epoch 168
Validation prc-auc = 0.441497
Epoch 169
Loss = 4.4580e-02, PNorm = 95.8312, GNorm = 1.0557, lr_0 = 7.5782e-04
Validation prc-auc = 0.440647
Epoch 170
Validation prc-auc = 0.437054
Epoch 171
Validation prc-auc = 0.431435
Epoch 172
Validation prc-auc = 0.424790
Epoch 173
Validation prc-auc = 0.398701
Epoch 174
Loss = 4.7713e-02, PNorm = 96.0291, GNorm = 0.8202, lr_0 = 7.5134e-04
Validation prc-auc = 0.404405
Epoch 175
Validation prc-auc = 0.409226
Epoch 176
Validation prc-auc = 0.409874
Epoch 177
Validation prc-auc = 0.430624
Epoch 178
Validation prc-auc = 0.413087
Epoch 179
Loss = 4.2253e-02, PNorm = 96.1781, GNorm = 0.4681, lr_0 = 7.4492e-04
Validation prc-auc = 0.412698
Epoch 180
Validation prc-auc = 0.408838
Epoch 181
Validation prc-auc = 0.407208
Epoch 182
Validation prc-auc = 0.409892
Epoch 183
Validation prc-auc = 0.405996
Epoch 184
Loss = 6.0650e-02, PNorm = 96.2945, GNorm = 0.0601, lr_0 = 7.3855e-04
Validation prc-auc = 0.423109
Epoch 185
Validation prc-auc = 0.422248
Epoch 186
Validation prc-auc = 0.422405
Epoch 187
Validation prc-auc = 0.419518
Epoch 188
Validation prc-auc = 0.398725
Epoch 189
Loss = 2.6848e-02, PNorm = 96.3827, GNorm = 1.1919, lr_0 = 7.3224e-04
Validation prc-auc = 0.402991
Epoch 190
Validation prc-auc = 0.391707
Epoch 191
Validation prc-auc = 0.404137
Epoch 192
Validation prc-auc = 0.423736
Epoch 193
Validation prc-auc = 0.417032
Epoch 194
Loss = 7.1110e-03, PNorm = 96.4870, GNorm = 0.1399, lr_0 = 7.2598e-04
Validation prc-auc = 0.409466
Epoch 195
Validation prc-auc = 0.406801
Epoch 196
Validation prc-auc = 0.407410
Epoch 197
Validation prc-auc = 0.399707
Epoch 198
Validation prc-auc = 0.391673
Epoch 199
Loss = 4.8912e-03, PNorm = 96.5740, GNorm = 0.0677, lr_0 = 7.1977e-04
Validation prc-auc = 0.392785
Epoch 200
Validation prc-auc = 0.392657
Epoch 201
Validation prc-auc = 0.401162
Epoch 202
Validation prc-auc = 0.423204
Epoch 203
Validation prc-auc = 0.422215
Epoch 204
Loss = 1.1910e-02, PNorm = 96.6524, GNorm = 0.0442, lr_0 = 7.1362e-04
Validation prc-auc = 0.424522
Epoch 205
Validation prc-auc = 0.423986
Epoch 206
Validation prc-auc = 0.428042
Epoch 207
Validation prc-auc = 0.426119
Epoch 208
Validation prc-auc = 0.420146
Epoch 209
Loss = 2.3681e-03, PNorm = 96.7277, GNorm = 0.0806, lr_0 = 7.0752e-04
Validation prc-auc = 0.418612
Epoch 210
Validation prc-auc = 0.421864
Epoch 211
Validation prc-auc = 0.439254
Epoch 212
Validation prc-auc = 0.431284
Epoch 213
Validation prc-auc = 0.431451
Epoch 214
Loss = 8.8794e-03, PNorm = 96.8067, GNorm = 0.5536, lr_0 = 7.0147e-04
Validation prc-auc = 0.422851
Epoch 215
Validation prc-auc = 0.421379
Epoch 216
Validation prc-auc = 0.435369
Epoch 217
Validation prc-auc = 0.434715
Epoch 218
Validation prc-auc = 0.435831
Epoch 219
Loss = 1.6831e-02, PNorm = 96.8733, GNorm = 1.1083, lr_0 = 6.9548e-04
Validation prc-auc = 0.435514
Epoch 220
Validation prc-auc = 0.419278
Epoch 221
Validation prc-auc = 0.418804
Epoch 222
Validation prc-auc = 0.418860
Epoch 223
Validation prc-auc = 0.418846
Epoch 224
Loss = 7.0522e-04, PNorm = 96.9256, GNorm = 0.0250, lr_0 = 6.8953e-04
Validation prc-auc = 0.426796
Epoch 225
Validation prc-auc = 0.426881
Epoch 226
Validation prc-auc = 0.419247
Epoch 227
Validation prc-auc = 0.419883
Epoch 228
Validation prc-auc = 0.420449
Epoch 229
Loss = 7.7741e-04, PNorm = 96.9741, GNorm = 0.0634, lr_0 = 6.8364e-04
Validation prc-auc = 0.421896
Epoch 230
Validation prc-auc = 0.422264
Epoch 231
Validation prc-auc = 0.422347
Epoch 232
Validation prc-auc = 0.430751
Epoch 233
Validation prc-auc = 0.430408
Epoch 234
Loss = 7.4928e-04, PNorm = 97.0101, GNorm = 0.0391, lr_0 = 6.7779e-04
Validation prc-auc = 0.430169
Epoch 235
Validation prc-auc = 0.429283
Epoch 236
Validation prc-auc = 0.428882
Epoch 237
Validation prc-auc = 0.428431
Epoch 238
Validation prc-auc = 0.428266
Epoch 239
Loss = 7.1551e-04, PNorm = 97.0397, GNorm = 0.0906, lr_0 = 6.7200e-04
Validation prc-auc = 0.427885
Epoch 240
Validation prc-auc = 0.427663
Epoch 241
Validation prc-auc = 0.427567
Epoch 242
Validation prc-auc = 0.417355
Epoch 243
Validation prc-auc = 0.415755
Epoch 244
Loss = 3.8225e-02, PNorm = 97.0624, GNorm = 0.0144, lr_0 = 6.6625e-04
Validation prc-auc = 0.415574
Epoch 245
Validation prc-auc = 0.415682
Epoch 246
Validation prc-auc = 0.415887
Epoch 247
Validation prc-auc = 0.415727
Epoch 248
Validation prc-auc = 0.415881
Epoch 249
Loss = 1.2586e-03, PNorm = 97.1068, GNorm = 0.1407, lr_0 = 6.6056e-04
Validation prc-auc = 0.424041
Epoch 250
Validation prc-auc = 0.424140
Epoch 251
Validation prc-auc = 0.424708
Epoch 252
Validation prc-auc = 0.424523
Epoch 253
Validation prc-auc = 0.424757
Epoch 254
Loss = 3.4111e-04, PNorm = 97.1481, GNorm = 0.0146, lr_0 = 6.5491e-04
Validation prc-auc = 0.425175
Epoch 255
Validation prc-auc = 0.425148
Epoch 256
Validation prc-auc = 0.425007
Epoch 257
Validation prc-auc = 0.425575
Epoch 258
Validation prc-auc = 0.426454
Epoch 259
Loss = 4.8427e-04, PNorm = 97.1759, GNorm = 0.0130, lr_0 = 6.4931e-04
Validation prc-auc = 0.427519
Epoch 260
Validation prc-auc = 0.427524
Epoch 261
Validation prc-auc = 0.426476
Epoch 262
Validation prc-auc = 0.425272
Epoch 263
Validation prc-auc = 0.431983
Epoch 264
Loss = 3.1558e-03, PNorm = 97.2228, GNorm = 0.0719, lr_0 = 6.4376e-04
Validation prc-auc = 0.425484
Epoch 265
Validation prc-auc = 0.420295
Epoch 266
Validation prc-auc = 0.402917
Epoch 267
Validation prc-auc = 0.398675
Epoch 268
Validation prc-auc = 0.398394
Epoch 269
Loss = 1.0604e-03, PNorm = 97.2910, GNorm = 0.0429, lr_0 = 6.3826e-04
Validation prc-auc = 0.397956
Epoch 270
Validation prc-auc = 0.398355
Epoch 271
Validation prc-auc = 0.402296
Epoch 272
Validation prc-auc = 0.402970
Epoch 273
Validation prc-auc = 0.418207
Epoch 274
Loss = 7.0823e-04, PNorm = 97.3408, GNorm = 0.0388, lr_0 = 6.3280e-04
Validation prc-auc = 0.418385
Epoch 275
Validation prc-auc = 0.418734
Epoch 276
Validation prc-auc = 0.424817
Epoch 277
Validation prc-auc = 0.424710
Epoch 278
Validation prc-auc = 0.425374
Epoch 279
Loss = 3.4757e-04, PNorm = 97.3721, GNorm = 0.0128, lr_0 = 6.2739e-04
Validation prc-auc = 0.425714
Epoch 280
Validation prc-auc = 0.426711
Epoch 281
Validation prc-auc = 0.428059
Epoch 282
Validation prc-auc = 0.427979
Epoch 283
Validation prc-auc = 0.418401
Epoch 284
Loss = 7.8947e-03, PNorm = 97.3989, GNorm = 0.5585, lr_0 = 6.2203e-04
Validation prc-auc = 0.417869
Epoch 285
Validation prc-auc = 0.417822
Epoch 286
Validation prc-auc = 0.418565
Epoch 287
Validation prc-auc = 0.418589
Epoch 288
Validation prc-auc = 0.418169
Epoch 289
Loss = 1.4315e-02, PNorm = 97.4258, GNorm = 1.3419, lr_0 = 6.1671e-04
Validation prc-auc = 0.427774
Epoch 290
Validation prc-auc = 0.428440
Epoch 291
Validation prc-auc = 0.427581
Epoch 292
Validation prc-auc = 0.421630
Epoch 293
Validation prc-auc = 0.422053
Epoch 294
Loss = 2.7856e-02, PNorm = 97.4956, GNorm = 1.6623, lr_0 = 6.1144e-04
Validation prc-auc = 0.423048
Epoch 295
Validation prc-auc = 0.436424
Epoch 296
Validation prc-auc = 0.433763
Epoch 297
Validation prc-auc = 0.434734
Epoch 298
Validation prc-auc = 0.432312
Epoch 299
Loss = 2.8712e-02, PNorm = 97.6089, GNorm = 0.0545, lr_0 = 6.0621e-04
Validation prc-auc = 0.435457
Model 0 best validation prc-auc = 0.464657 on epoch 64
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.315661
Ensemble test prc-auc = 0.315661
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.347106
Epoch 1
Validation prc-auc = 0.366081
Epoch 2
Validation prc-auc = 0.313646
Epoch 3
Validation prc-auc = 0.141834
Epoch 4
Loss = 6.4512e-01, PNorm = 87.5311, GNorm = 2.6050, lr_0 = 6.5000e-04
Validation prc-auc = 0.272246
Epoch 5
Validation prc-auc = 0.415886
Epoch 6
Validation prc-auc = 0.399762
Epoch 7
Validation prc-auc = 0.425047
Epoch 8
Validation prc-auc = 0.187754
Epoch 9
Loss = 4.8831e-01, PNorm = 87.8780, GNorm = 1.4430, lr_0 = 9.9743e-04
Validation prc-auc = 0.168753
Epoch 10
Validation prc-auc = 0.344885
Epoch 11
Validation prc-auc = 0.429660
Epoch 12
Validation prc-auc = 0.407981
Epoch 13
Validation prc-auc = 0.309257
Epoch 14
Loss = 3.3462e-01, PNorm = 88.2501, GNorm = 1.1897, lr_0 = 9.8890e-04
Validation prc-auc = 0.312871
Epoch 15
Validation prc-auc = 0.338372
Epoch 16
Validation prc-auc = 0.398388
Epoch 17
Validation prc-auc = 0.426172
Epoch 18
Validation prc-auc = 0.352609
Epoch 19
Loss = 3.1063e-01, PNorm = 88.5139, GNorm = 1.2558, lr_0 = 9.8045e-04
Validation prc-auc = 0.194192
Epoch 20
Validation prc-auc = 0.341152
Epoch 21
Validation prc-auc = 0.429105
Epoch 22
Validation prc-auc = 0.372277
Epoch 23
Validation prc-auc = 0.326878
Epoch 24
Loss = 2.4832e-01, PNorm = 88.7236, GNorm = 0.5435, lr_0 = 9.7207e-04
Validation prc-auc = 0.409945
Epoch 25
Validation prc-auc = 0.389436
Epoch 26
Validation prc-auc = 0.365667
Epoch 27
Validation prc-auc = 0.349014
Epoch 28
Validation prc-auc = 0.351662
Epoch 29
Loss = 2.9670e-01, PNorm = 88.9044, GNorm = 1.6247, lr_0 = 9.6376e-04
Validation prc-auc = 0.411767
Epoch 30
Validation prc-auc = 0.374054
Epoch 31
Validation prc-auc = 0.284593
Epoch 32
Validation prc-auc = 0.309402
Epoch 33
Validation prc-auc = 0.335937
Epoch 34
Loss = 3.2649e-01, PNorm = 89.0847, GNorm = 0.8434, lr_0 = 9.5552e-04
Validation prc-auc = 0.297063
Epoch 35
Validation prc-auc = 0.283955
Epoch 36
Validation prc-auc = 0.366764
Epoch 37
Validation prc-auc = 0.439448
Epoch 38
Validation prc-auc = 0.447859
Epoch 39
Loss = 1.8100e-01, PNorm = 89.2641, GNorm = 1.6665, lr_0 = 9.4735e-04
Validation prc-auc = 0.426565
Epoch 40
Validation prc-auc = 0.389632
Epoch 41
Validation prc-auc = 0.383229
Epoch 42
Validation prc-auc = 0.384328
Epoch 43
Validation prc-auc = 0.396005
Epoch 44
Loss = 2.1492e-01, PNorm = 89.4588, GNorm = 1.8342, lr_0 = 9.3925e-04
Validation prc-auc = 0.399575
Epoch 45
Validation prc-auc = 0.420414
Epoch 46
Validation prc-auc = 0.409136
Epoch 47
Validation prc-auc = 0.395643
Epoch 48
Validation prc-auc = 0.396587
Epoch 49
Loss = 1.3351e-01, PNorm = 89.6650, GNorm = 1.1945, lr_0 = 9.3122e-04
Validation prc-auc = 0.359562
Epoch 50
Validation prc-auc = 0.433989
Epoch 51
Validation prc-auc = 0.438654
Epoch 52
Validation prc-auc = 0.438178
Epoch 53
Validation prc-auc = 0.274479
Epoch 54
Loss = 2.6755e-01, PNorm = 89.8797, GNorm = 2.7999, lr_0 = 9.2326e-04
Validation prc-auc = 0.383597
Epoch 55
Validation prc-auc = 0.448116
Epoch 56
Validation prc-auc = 0.455549
Epoch 57
Validation prc-auc = 0.448732
Epoch 58
Validation prc-auc = 0.433275
Epoch 59
Loss = 2.6936e-01, PNorm = 90.1300, GNorm = 1.1574, lr_0 = 9.1537e-04
Validation prc-auc = 0.281926
Epoch 60
Validation prc-auc = 0.252228
Epoch 61
Validation prc-auc = 0.388786
Epoch 62
Validation prc-auc = 0.428806
Epoch 63
Validation prc-auc = 0.444523
Epoch 64
Loss = 1.8256e-01, PNorm = 90.3642, GNorm = 2.1577, lr_0 = 9.0754e-04
Validation prc-auc = 0.454638
Epoch 65
Validation prc-auc = 0.453047
Epoch 66
Validation prc-auc = 0.445204
Epoch 67
Validation prc-auc = 0.413862
Epoch 68
Validation prc-auc = 0.378940
Epoch 69
Loss = 1.2616e-01, PNorm = 90.5932, GNorm = 1.4907, lr_0 = 8.9978e-04
Validation prc-auc = 0.392475
Epoch 70
Validation prc-auc = 0.464387
Epoch 71
Validation prc-auc = 0.458514
Epoch 72
Validation prc-auc = 0.463174
Epoch 73
Validation prc-auc = 0.458454
Epoch 74
Loss = 1.4412e-01, PNorm = 90.8430, GNorm = 1.5181, lr_0 = 8.9209e-04
Validation prc-auc = 0.449067
Epoch 75
Validation prc-auc = 0.388424
Epoch 76
Validation prc-auc = 0.412811
Epoch 77
Validation prc-auc = 0.420797
Epoch 78
Validation prc-auc = 0.463595
Epoch 79
Loss = 1.6192e-01, PNorm = 91.1295, GNorm = 3.8352, lr_0 = 8.8447e-04
Validation prc-auc = 0.454253
Epoch 80
Validation prc-auc = 0.413393
Epoch 81
Validation prc-auc = 0.443548
Epoch 82
Validation prc-auc = 0.414180
Epoch 83
Validation prc-auc = 0.220250
Epoch 84
Loss = 2.5315e-01, PNorm = 91.4352, GNorm = 1.7026, lr_0 = 8.7691e-04
Validation prc-auc = 0.349861
Epoch 85
Validation prc-auc = 0.409774
Epoch 86
Validation prc-auc = 0.436455
Epoch 87
Validation prc-auc = 0.439807
Epoch 88
Validation prc-auc = 0.415769
Epoch 89
Loss = 1.0457e-01, PNorm = 91.7266, GNorm = 1.4310, lr_0 = 8.6941e-04
Validation prc-auc = 0.405230
Epoch 90
Validation prc-auc = 0.397398
Epoch 91
Validation prc-auc = 0.415286
Epoch 92
Validation prc-auc = 0.415205
Epoch 93
Validation prc-auc = 0.429663
Epoch 94
Loss = 8.2886e-02, PNorm = 91.9416, GNorm = 1.0167, lr_0 = 8.6198e-04
Validation prc-auc = 0.426018
Epoch 95
Validation prc-auc = 0.422326
Epoch 96
Validation prc-auc = 0.427330
Epoch 97
Validation prc-auc = 0.421526
Epoch 98
Validation prc-auc = 0.424932
Epoch 99
Loss = 1.0915e-01, PNorm = 92.1245, GNorm = 2.1196, lr_0 = 8.5461e-04
Validation prc-auc = 0.435547
Epoch 100
Validation prc-auc = 0.394831
Epoch 101
Validation prc-auc = 0.390392
Epoch 102
Validation prc-auc = 0.397463
Epoch 103
Validation prc-auc = 0.395278
Epoch 104
Loss = 5.3250e-02, PNorm = 92.3176, GNorm = 0.7491, lr_0 = 8.4730e-04
Validation prc-auc = 0.424663
Epoch 105
Validation prc-auc = 0.445886
Epoch 106
Validation prc-auc = 0.449160
Epoch 107
Validation prc-auc = 0.438884
Epoch 108
Validation prc-auc = 0.435683
Epoch 109
Loss = 4.9781e-02, PNorm = 92.4937, GNorm = 1.3881, lr_0 = 8.4006e-04
Validation prc-auc = 0.407991
Epoch 110
Validation prc-auc = 0.380779
Epoch 111
Validation prc-auc = 0.404697
Epoch 112
Validation prc-auc = 0.453371
Epoch 113
Validation prc-auc = 0.464728
Epoch 114
Loss = 7.7243e-02, PNorm = 92.6890, GNorm = 0.3055, lr_0 = 8.3288e-04
Validation prc-auc = 0.438925
Epoch 115
Validation prc-auc = 0.449955
Epoch 116
Validation prc-auc = 0.422907
Epoch 117
Validation prc-auc = 0.366056
Epoch 118
Validation prc-auc = 0.354580
Epoch 119
Loss = 3.2369e-02, PNorm = 92.9171, GNorm = 0.1786, lr_0 = 8.2576e-04
Validation prc-auc = 0.367308
Epoch 120
Validation prc-auc = 0.416393
Epoch 121
Validation prc-auc = 0.430012
Epoch 122
Validation prc-auc = 0.415145
Epoch 123
Validation prc-auc = 0.408615
Epoch 124
Loss = 2.7188e-02, PNorm = 93.1216, GNorm = 0.6101, lr_0 = 8.1870e-04
Validation prc-auc = 0.397431
Epoch 125
Validation prc-auc = 0.415642
Epoch 126
Validation prc-auc = 0.362442
Epoch 127
Validation prc-auc = 0.324240
Epoch 128
Validation prc-auc = 0.303511
Epoch 129
Loss = 1.0625e-01, PNorm = 93.3543, GNorm = 1.3923, lr_0 = 8.1170e-04
Validation prc-auc = 0.310375
Epoch 130
Validation prc-auc = 0.393481
Epoch 131
Validation prc-auc = 0.433341
Epoch 132
Validation prc-auc = 0.393451
Epoch 133
Validation prc-auc = 0.383046
Epoch 134
Loss = 5.4710e-02, PNorm = 93.5459, GNorm = 0.3601, lr_0 = 8.0476e-04
Validation prc-auc = 0.415087
Epoch 135
Validation prc-auc = 0.421535
Epoch 136
Validation prc-auc = 0.425290
Epoch 137
Validation prc-auc = 0.429825
Epoch 138
Validation prc-auc = 0.413018
Epoch 139
Loss = 6.5380e-02, PNorm = 93.7203, GNorm = 1.4287, lr_0 = 7.9788e-04
Validation prc-auc = 0.425603
Epoch 140
Validation prc-auc = 0.411610
Epoch 141
Validation prc-auc = 0.418893
Epoch 142
Validation prc-auc = 0.444403
Epoch 143
Validation prc-auc = 0.448270
Epoch 144
Loss = 9.2475e-02, PNorm = 93.9021, GNorm = 0.8673, lr_0 = 7.9106e-04
Validation prc-auc = 0.461479
Epoch 145
Validation prc-auc = 0.456877
Epoch 146
Validation prc-auc = 0.466393
Epoch 147
Validation prc-auc = 0.456827
Epoch 148
Validation prc-auc = 0.428467
Epoch 149
Loss = 4.6811e-02, PNorm = 94.0984, GNorm = 0.4026, lr_0 = 7.8430e-04
Validation prc-auc = 0.415707
Epoch 150
Validation prc-auc = 0.325550
Epoch 151
Validation prc-auc = 0.263768
Epoch 152
Validation prc-auc = 0.361237
Epoch 153
Validation prc-auc = 0.422674
Epoch 154
Loss = 1.1901e-02, PNorm = 94.2775, GNorm = 0.2065, lr_0 = 7.7759e-04
Validation prc-auc = 0.429778
Epoch 155
Validation prc-auc = 0.455303
Epoch 156
Validation prc-auc = 0.463731
Epoch 157
Validation prc-auc = 0.465844
Epoch 158
Validation prc-auc = 0.457754
Epoch 159
Loss = 2.7466e-02, PNorm = 94.4465, GNorm = 1.0216, lr_0 = 7.7095e-04
Validation prc-auc = 0.454750
Epoch 160
Validation prc-auc = 0.452169
Epoch 161
Validation prc-auc = 0.461563
Epoch 162
Validation prc-auc = 0.460690
Epoch 163
Validation prc-auc = 0.443362
Epoch 164
Loss = 5.2788e-02, PNorm = 94.6053, GNorm = 2.1823, lr_0 = 7.6436e-04
Validation prc-auc = 0.450939
Epoch 165
Validation prc-auc = 0.464155
Epoch 166
Validation prc-auc = 0.456535
Epoch 167
Validation prc-auc = 0.439179
Epoch 168
Validation prc-auc = 0.438164
Epoch 169
Loss = 3.0434e-02, PNorm = 94.7379, GNorm = 0.4300, lr_0 = 7.5782e-04
Validation prc-auc = 0.455765
Epoch 170
Validation prc-auc = 0.438887
Epoch 171
Validation prc-auc = 0.444524
Epoch 172
Validation prc-auc = 0.457666
Epoch 173
Validation prc-auc = 0.458340
Epoch 174
Loss = 7.7941e-02, PNorm = 94.8787, GNorm = 2.3115, lr_0 = 7.5134e-04
Validation prc-auc = 0.458470
Epoch 175
Validation prc-auc = 0.454080
Epoch 176
Validation prc-auc = 0.450715
Epoch 177
Validation prc-auc = 0.441552
Epoch 178
Validation prc-auc = 0.419965
Epoch 179
Loss = 6.8634e-02, PNorm = 95.0203, GNorm = 0.9756, lr_0 = 7.4492e-04
Validation prc-auc = 0.414532
Epoch 180
Validation prc-auc = 0.406619
Epoch 181
Validation prc-auc = 0.393918
Epoch 182
Validation prc-auc = 0.335022
Epoch 183
Validation prc-auc = 0.380836
Epoch 184
Loss = 1.7387e-02, PNorm = 95.2283, GNorm = 0.1941, lr_0 = 7.3855e-04
Validation prc-auc = 0.391052
Epoch 185
Validation prc-auc = 0.394478
Epoch 186
Validation prc-auc = 0.391472
Epoch 187
Validation prc-auc = 0.384058
Epoch 188
Validation prc-auc = 0.382440
Epoch 189
Loss = 5.2804e-02, PNorm = 95.4464, GNorm = 0.4208, lr_0 = 7.3224e-04
Validation prc-auc = 0.391146
Epoch 190
Validation prc-auc = 0.395247
Epoch 191
Validation prc-auc = 0.412130
Epoch 192
Validation prc-auc = 0.420973
Epoch 193
Validation prc-auc = 0.417111
Epoch 194
Loss = 5.9620e-03, PNorm = 95.6334, GNorm = 0.3162, lr_0 = 7.2598e-04
Validation prc-auc = 0.415696
Epoch 195
Validation prc-auc = 0.420242
Epoch 196
Validation prc-auc = 0.418981
Epoch 197
Validation prc-auc = 0.406712
Epoch 198
Validation prc-auc = 0.402133
Epoch 199
Loss = 1.5505e-02, PNorm = 95.7561, GNorm = 0.9580, lr_0 = 7.1977e-04
Validation prc-auc = 0.399505
Epoch 200
Validation prc-auc = 0.399095
Epoch 201
Validation prc-auc = 0.399289
Epoch 202
Validation prc-auc = 0.405285
Epoch 203
Validation prc-auc = 0.409826
Epoch 204
Loss = 2.0317e-03, PNorm = 95.8450, GNorm = 0.0449, lr_0 = 7.1362e-04
Validation prc-auc = 0.411303
Epoch 205
Validation prc-auc = 0.411203
Epoch 206
Validation prc-auc = 0.422088
Epoch 207
Validation prc-auc = 0.417188
Epoch 208
Validation prc-auc = 0.417375
Epoch 209
Loss = 2.4834e-03, PNorm = 95.9057, GNorm = 0.0487, lr_0 = 7.0752e-04
Validation prc-auc = 0.415282
Epoch 210
Validation prc-auc = 0.414536
Epoch 211
Validation prc-auc = 0.409585
Epoch 212
Validation prc-auc = 0.411820
Epoch 213
Validation prc-auc = 0.412285
Epoch 214
Loss = 1.5082e-02, PNorm = 95.9786, GNorm = 0.1477, lr_0 = 7.0147e-04
Validation prc-auc = 0.407626
Epoch 215
Validation prc-auc = 0.408816
Epoch 216
Validation prc-auc = 0.412765
Epoch 217
Validation prc-auc = 0.414784
Epoch 218
Validation prc-auc = 0.419900
Epoch 219
Loss = 3.2938e-02, PNorm = 96.0615, GNorm = 0.0013, lr_0 = 6.9548e-04
Validation prc-auc = 0.425528
Epoch 220
Validation prc-auc = 0.423667
Epoch 221
Validation prc-auc = 0.423445
Epoch 222
Validation prc-auc = 0.423397
Epoch 223
Validation prc-auc = 0.429615
Epoch 224
Loss = 2.2064e-03, PNorm = 96.1555, GNorm = 0.0775, lr_0 = 6.8953e-04
Validation prc-auc = 0.425874
Epoch 225
Validation prc-auc = 0.417387
Epoch 226
Validation prc-auc = 0.428139
Epoch 227
Validation prc-auc = 0.409296
Epoch 228
Validation prc-auc = 0.436135
Epoch 229
Loss = 6.6269e-02, PNorm = 96.2614, GNorm = 0.1465, lr_0 = 6.8364e-04
Validation prc-auc = 0.436683
Epoch 230
Validation prc-auc = 0.426431
Epoch 231
Validation prc-auc = 0.430253
Epoch 232
Validation prc-auc = 0.424323
Epoch 233
Validation prc-auc = 0.412548
Epoch 234
Loss = 7.4602e-02, PNorm = 96.4227, GNorm = 0.6659, lr_0 = 6.7779e-04
Validation prc-auc = 0.410971
Epoch 235
Validation prc-auc = 0.425691
Epoch 236
Validation prc-auc = 0.406101
Epoch 237
Validation prc-auc = 0.397404
Epoch 238
Validation prc-auc = 0.390985
Epoch 239
Loss = 3.4423e-03, PNorm = 96.6166, GNorm = 0.2484, lr_0 = 6.7200e-04
Validation prc-auc = 0.388894
Epoch 240
Validation prc-auc = 0.390314
Epoch 241
Validation prc-auc = 0.406131
Epoch 242
Validation prc-auc = 0.408263
Epoch 243
Validation prc-auc = 0.410550
Epoch 244
Loss = 5.9083e-02, PNorm = 96.8645, GNorm = 1.3963, lr_0 = 6.6625e-04
Validation prc-auc = 0.416348
Epoch 245
Validation prc-auc = 0.416748
Epoch 246
Validation prc-auc = 0.412039
Epoch 247
Validation prc-auc = 0.409498
Epoch 248
Validation prc-auc = 0.420163
Epoch 249
Loss = 9.0980e-03, PNorm = 97.0695, GNorm = 0.0754, lr_0 = 6.6056e-04
Validation prc-auc = 0.421595
Epoch 250
Validation prc-auc = 0.425216
Epoch 251
Validation prc-auc = 0.429915
Epoch 252
Validation prc-auc = 0.435714
Epoch 253
Validation prc-auc = 0.452024
Epoch 254
Loss = 8.0329e-03, PNorm = 97.2250, GNorm = 0.4788, lr_0 = 6.5491e-04
Validation prc-auc = 0.439772
Epoch 255
Validation prc-auc = 0.440131
Epoch 256
Validation prc-auc = 0.419238
Epoch 257
Validation prc-auc = 0.422721
Epoch 258
Validation prc-auc = 0.432280
Epoch 259
Loss = 1.1827e-02, PNorm = 97.3455, GNorm = 0.3376, lr_0 = 6.4931e-04
Validation prc-auc = 0.429718
Epoch 260
Validation prc-auc = 0.420114
Epoch 261
Validation prc-auc = 0.404597
Epoch 262
Validation prc-auc = 0.362775
Epoch 263
Validation prc-auc = 0.275477
Epoch 264
Loss = 1.3701e-02, PNorm = 97.4520, GNorm = 0.3775, lr_0 = 6.4376e-04
Validation prc-auc = 0.273989
Epoch 265
Validation prc-auc = 0.292031
Epoch 266
Validation prc-auc = 0.407302
Epoch 267
Validation prc-auc = 0.439527
Epoch 268
Validation prc-auc = 0.444855
Epoch 269
Loss = 1.6990e-02, PNorm = 97.5895, GNorm = 0.7114, lr_0 = 6.3826e-04
Validation prc-auc = 0.448889
Epoch 270
Validation prc-auc = 0.448275
Epoch 271
Validation prc-auc = 0.448419
Epoch 272
Validation prc-auc = 0.444683
Epoch 273
Validation prc-auc = 0.447387
Epoch 274
Loss = 1.5159e-03, PNorm = 97.7417, GNorm = 0.1081, lr_0 = 6.3280e-04
Validation prc-auc = 0.439840
Epoch 275
Validation prc-auc = 0.428497
Epoch 276
Validation prc-auc = 0.432397
Epoch 277
Validation prc-auc = 0.440357
Epoch 278
Validation prc-auc = 0.459040
Epoch 279
Loss = 2.1058e-02, PNorm = 97.8319, GNorm = 0.0547, lr_0 = 6.2739e-04
Validation prc-auc = 0.459412
Epoch 280
Validation prc-auc = 0.458640
Epoch 281
Validation prc-auc = 0.454087
Epoch 282
Validation prc-auc = 0.438583
Epoch 283
Validation prc-auc = 0.434574
Epoch 284
Loss = 2.2908e-03, PNorm = 97.9150, GNorm = 0.0179, lr_0 = 6.2203e-04
Validation prc-auc = 0.436477
Epoch 285
Validation prc-auc = 0.417538
Epoch 286
Validation prc-auc = 0.418207
Epoch 287
Validation prc-auc = 0.424213
Epoch 288
Validation prc-auc = 0.426315
Epoch 289
Loss = 2.4407e-02, PNorm = 98.0175, GNorm = 1.1540, lr_0 = 6.1671e-04
Validation prc-auc = 0.427397
Epoch 290
Validation prc-auc = 0.424916
Epoch 291
Validation prc-auc = 0.425125
Epoch 292
Validation prc-auc = 0.418763
Epoch 293
Validation prc-auc = 0.426615
Epoch 294
Loss = 6.5540e-03, PNorm = 98.1459, GNorm = 0.7617, lr_0 = 6.1144e-04
Validation prc-auc = 0.426351
Epoch 295
Validation prc-auc = 0.419173
Epoch 296
Validation prc-auc = 0.437598
Epoch 297
Validation prc-auc = 0.427139
Epoch 298
Validation prc-auc = 0.432666
Epoch 299
Loss = 1.2008e-02, PNorm = 98.2317, GNorm = 0.7357, lr_0 = 6.0621e-04
Validation prc-auc = 0.418744
Model 0 best validation prc-auc = 0.466393 on epoch 146
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.244857
Ensemble test prc-auc = 0.244857
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.361053
Epoch 1
Validation prc-auc = 0.341266
Epoch 2
Validation prc-auc = 0.369776
Epoch 3
Validation prc-auc = 0.168239
Epoch 4
Loss = 5.9851e-01, PNorm = 87.5360, GNorm = 1.9357, lr_0 = 6.5000e-04
Validation prc-auc = 0.342976
Epoch 5
Validation prc-auc = 0.389251
Epoch 6
Validation prc-auc = 0.369428
Epoch 7
Validation prc-auc = 0.337748
Epoch 8
Validation prc-auc = 0.165839
Epoch 9
Loss = 4.7904e-01, PNorm = 87.8958, GNorm = 2.3665, lr_0 = 9.9743e-04
Validation prc-auc = 0.329819
Epoch 10
Validation prc-auc = 0.410894
Epoch 11
Validation prc-auc = 0.386398
Epoch 12
Validation prc-auc = 0.305202
Epoch 13
Validation prc-auc = 0.360685
Epoch 14
Loss = 5.4907e-01, PNorm = 88.2819, GNorm = 2.7752, lr_0 = 9.8890e-04
Validation prc-auc = 0.376837
Epoch 15
Validation prc-auc = 0.310132
Epoch 16
Validation prc-auc = 0.327394
Epoch 17
Validation prc-auc = 0.367334
Epoch 18
Validation prc-auc = 0.349245
Epoch 19
Loss = 3.9934e-01, PNorm = 88.5718, GNorm = 1.9826, lr_0 = 9.8045e-04
Validation prc-auc = 0.386339
Epoch 20
Validation prc-auc = 0.326719
Epoch 21
Validation prc-auc = 0.418675
Epoch 22
Validation prc-auc = 0.378662
Epoch 23
Validation prc-auc = 0.278336
Epoch 24
Loss = 3.7268e-01, PNorm = 88.8077, GNorm = 2.7952, lr_0 = 9.7207e-04
Validation prc-auc = 0.316170
Epoch 25
Validation prc-auc = 0.370902
Epoch 26
Validation prc-auc = 0.375028
Epoch 27
Validation prc-auc = 0.283146
Epoch 28
Validation prc-auc = 0.340637
Epoch 29
Loss = 3.1349e-01, PNorm = 89.0349, GNorm = 1.5599, lr_0 = 9.6376e-04
Validation prc-auc = 0.384379
Epoch 30
Validation prc-auc = 0.439581
Epoch 31
Validation prc-auc = 0.432290
Epoch 32
Validation prc-auc = 0.410512
Epoch 33
Validation prc-auc = 0.353080
Epoch 34
Loss = 4.3693e-01, PNorm = 89.2661, GNorm = 1.9502, lr_0 = 9.5552e-04
Validation prc-auc = 0.434449
Epoch 35
Validation prc-auc = 0.373627
Epoch 36
Validation prc-auc = 0.358102
Epoch 37
Validation prc-auc = 0.320547
Epoch 38
Validation prc-auc = 0.408710
Epoch 39
Loss = 2.4312e-01, PNorm = 89.4774, GNorm = 1.6201, lr_0 = 9.4735e-04
Validation prc-auc = 0.422052
Epoch 40
Validation prc-auc = 0.433959
Epoch 41
Validation prc-auc = 0.372439
Epoch 42
Validation prc-auc = 0.275488
Epoch 43
Validation prc-auc = 0.362819
Epoch 44
Loss = 3.6106e-01, PNorm = 89.6919, GNorm = 1.1855, lr_0 = 9.3925e-04
Validation prc-auc = 0.459034
Epoch 45
Validation prc-auc = 0.456169
Epoch 46
Validation prc-auc = 0.440990
Epoch 47
Validation prc-auc = 0.430128
Epoch 48
Validation prc-auc = 0.430301
Epoch 49
Loss = 1.3264e-01, PNorm = 89.9133, GNorm = 0.7976, lr_0 = 9.3122e-04
Validation prc-auc = 0.440246
Epoch 50
Validation prc-auc = 0.433363
Epoch 51
Validation prc-auc = 0.434583
Epoch 52
Validation prc-auc = 0.364927
Epoch 53
Validation prc-auc = 0.343733
Epoch 54
Loss = 2.1085e-01, PNorm = 90.1406, GNorm = 2.0917, lr_0 = 9.2326e-04
Validation prc-auc = 0.408939
Epoch 55
Validation prc-auc = 0.442509
Epoch 56
Validation prc-auc = 0.445052
Epoch 57
Validation prc-auc = 0.441499
Epoch 58
Validation prc-auc = 0.416548
Epoch 59
Loss = 1.5649e-01, PNorm = 90.3800, GNorm = 2.7201, lr_0 = 9.1537e-04
Validation prc-auc = 0.403319
Epoch 60
Validation prc-auc = 0.404039
Epoch 61
Validation prc-auc = 0.400330
Epoch 62
Validation prc-auc = 0.436284
Epoch 63
Validation prc-auc = 0.454238
Epoch 64
Loss = 6.5103e-02, PNorm = 90.6484, GNorm = 0.7427, lr_0 = 9.0754e-04
Validation prc-auc = 0.450292
Epoch 65
Validation prc-auc = 0.456465
Epoch 66
Validation prc-auc = 0.460633
Epoch 67
Validation prc-auc = 0.440580
Epoch 68
Validation prc-auc = 0.437303
Epoch 69
Loss = 1.3025e-01, PNorm = 90.9151, GNorm = 2.0743, lr_0 = 8.9978e-04
Validation prc-auc = 0.450980
Epoch 70
Validation prc-auc = 0.444936
Epoch 71
Validation prc-auc = 0.462322
Epoch 72
Validation prc-auc = 0.443099
Epoch 73
Validation prc-auc = 0.440167
Epoch 74
Loss = 5.1531e-02, PNorm = 91.1978, GNorm = 0.6583, lr_0 = 8.9209e-04
Validation prc-auc = 0.413520
Epoch 75
Validation prc-auc = 0.410937
Epoch 76
Validation prc-auc = 0.440422
Epoch 77
Validation prc-auc = 0.404007
Epoch 78
Validation prc-auc = 0.391842
Epoch 79
Loss = 1.9235e-01, PNorm = 91.4761, GNorm = 2.9069, lr_0 = 8.8447e-04
Validation prc-auc = 0.451125
Epoch 80
Validation prc-auc = 0.445455
Epoch 81
Validation prc-auc = 0.449180
Epoch 82
Validation prc-auc = 0.476752
Epoch 83
Validation prc-auc = 0.485699
Epoch 84
Loss = 1.3901e-01, PNorm = 91.7617, GNorm = 1.6077, lr_0 = 8.7691e-04
Validation prc-auc = 0.485053
Epoch 85
Validation prc-auc = 0.461263
Epoch 86
Validation prc-auc = 0.421791
Epoch 87
Validation prc-auc = 0.438678
Epoch 88
Validation prc-auc = 0.439071
Epoch 89
Loss = 1.6025e-01, PNorm = 92.0490, GNorm = 1.3677, lr_0 = 8.6941e-04
Validation prc-auc = 0.441420
Epoch 90
Validation prc-auc = 0.415381
Epoch 91
Validation prc-auc = 0.430352
Epoch 92
Validation prc-auc = 0.445696
Epoch 93
Validation prc-auc = 0.450505
Epoch 94
Loss = 1.9983e-01, PNorm = 92.3496, GNorm = 2.0435, lr_0 = 8.6198e-04
Validation prc-auc = 0.434513
Epoch 95
Validation prc-auc = 0.443685
Epoch 96
Validation prc-auc = 0.449147
Epoch 97
Validation prc-auc = 0.453948
Epoch 98
Validation prc-auc = 0.428968
Epoch 99
Loss = 2.1605e-01, PNorm = 92.6409, GNorm = 1.9277, lr_0 = 8.5461e-04
Validation prc-auc = 0.422362
Epoch 100
Validation prc-auc = 0.223173
Epoch 101
Validation prc-auc = 0.176955
Epoch 102
Validation prc-auc = 0.250710
Epoch 103
Validation prc-auc = 0.395665
Epoch 104
Loss = 2.0366e-01, PNorm = 92.9639, GNorm = 1.7139, lr_0 = 8.4730e-04
Validation prc-auc = 0.416973
Epoch 105
Validation prc-auc = 0.439062
Epoch 106
Validation prc-auc = 0.446693
Epoch 107
Validation prc-auc = 0.460446
Epoch 108
Validation prc-auc = 0.444196
Epoch 109
Loss = 9.1091e-02, PNorm = 93.2766, GNorm = 0.4872, lr_0 = 8.4006e-04
Validation prc-auc = 0.408451
Epoch 110
Validation prc-auc = 0.408877
Epoch 111
Validation prc-auc = 0.413721
Epoch 112
Validation prc-auc = 0.429533
Epoch 113
Validation prc-auc = 0.442847
Epoch 114
Loss = 1.6186e-01, PNorm = 93.5518, GNorm = 1.3081, lr_0 = 8.3288e-04
Validation prc-auc = 0.441253
Epoch 115
Validation prc-auc = 0.436194
Epoch 116
Validation prc-auc = 0.407209
Epoch 117
Validation prc-auc = 0.397182
Epoch 118
Validation prc-auc = 0.418345
Epoch 119
Loss = 5.8737e-02, PNorm = 93.8080, GNorm = 0.5929, lr_0 = 8.2576e-04
Validation prc-auc = 0.411584
Epoch 120
Validation prc-auc = 0.428734
Epoch 121
Validation prc-auc = 0.429638
Epoch 122
Validation prc-auc = 0.441154
Epoch 123
Validation prc-auc = 0.447521
Epoch 124
Loss = 4.1868e-02, PNorm = 94.0097, GNorm = 0.6172, lr_0 = 8.1870e-04
Validation prc-auc = 0.436538
Epoch 125
Validation prc-auc = 0.454267
Epoch 126
Validation prc-auc = 0.454931
Epoch 127
Validation prc-auc = 0.453303
Epoch 128
Validation prc-auc = 0.456263
Epoch 129
Loss = 2.0888e-02, PNorm = 94.1556, GNorm = 0.4786, lr_0 = 8.1170e-04
Validation prc-auc = 0.465351
Epoch 130
Validation prc-auc = 0.453498
Epoch 131
Validation prc-auc = 0.461951
Epoch 132
Validation prc-auc = 0.451201
Epoch 133
Validation prc-auc = 0.453428
Epoch 134
Loss = 1.6636e-02, PNorm = 94.3093, GNorm = 0.5164, lr_0 = 8.0476e-04
Validation prc-auc = 0.445327
Epoch 135
Validation prc-auc = 0.419230
Epoch 136
Validation prc-auc = 0.409019
Epoch 137
Validation prc-auc = 0.391751
Epoch 138
Validation prc-auc = 0.387718
Epoch 139
Loss = 6.2196e-02, PNorm = 94.4856, GNorm = 1.3187, lr_0 = 7.9788e-04
Validation prc-auc = 0.385936
Epoch 140
Validation prc-auc = 0.386300
Epoch 141
Validation prc-auc = 0.414764
Epoch 142
Validation prc-auc = 0.429014
Epoch 143
Validation prc-auc = 0.447870
Epoch 144
Loss = 4.0551e-02, PNorm = 94.6492, GNorm = 0.3692, lr_0 = 7.9106e-04
Validation prc-auc = 0.455790
Epoch 145
Validation prc-auc = 0.459908
Epoch 146
Validation prc-auc = 0.455511
Epoch 147
Validation prc-auc = 0.438726
Epoch 148
Validation prc-auc = 0.436939
Epoch 149
Loss = 1.3321e-01, PNorm = 94.7999, GNorm = 5.3731, lr_0 = 7.8430e-04
Validation prc-auc = 0.444814
Epoch 150
Validation prc-auc = 0.435586
Epoch 151
Validation prc-auc = 0.426553
Epoch 152
Validation prc-auc = 0.431701
Epoch 153
Validation prc-auc = 0.430435
Epoch 154
Loss = 4.8254e-02, PNorm = 95.0073, GNorm = 1.3591, lr_0 = 7.7759e-04
Validation prc-auc = 0.463681
Epoch 155
Validation prc-auc = 0.450430
Epoch 156
Validation prc-auc = 0.446776
Epoch 157
Validation prc-auc = 0.469061
Epoch 158
Validation prc-auc = 0.463361
Epoch 159
Loss = 2.7268e-02, PNorm = 95.2472, GNorm = 1.0962, lr_0 = 7.7095e-04
Validation prc-auc = 0.474088
Epoch 160
Validation prc-auc = 0.465413
Epoch 161
Validation prc-auc = 0.447851
Epoch 162
Validation prc-auc = 0.455821
Epoch 163
Validation prc-auc = 0.444722
Epoch 164
Loss = 6.8195e-02, PNorm = 95.4638, GNorm = 1.2117, lr_0 = 7.6436e-04
Validation prc-auc = 0.439236
Epoch 165
Validation prc-auc = 0.453237
Epoch 166
Validation prc-auc = 0.458872
Epoch 167
Validation prc-auc = 0.460309
Epoch 168
Validation prc-auc = 0.456483
Epoch 169
Loss = 3.6851e-02, PNorm = 95.6333, GNorm = 0.6238, lr_0 = 7.5782e-04
Validation prc-auc = 0.434327
Epoch 170
Validation prc-auc = 0.435444
Epoch 171
Validation prc-auc = 0.444249
Epoch 172
Validation prc-auc = 0.457406
Epoch 173
Validation prc-auc = 0.439797
Epoch 174
Loss = 4.8218e-02, PNorm = 95.7606, GNorm = 0.8323, lr_0 = 7.5134e-04
Validation prc-auc = 0.449719
Epoch 175
Validation prc-auc = 0.450869
Epoch 176
Validation prc-auc = 0.454519
Epoch 177
Validation prc-auc = 0.454884
Epoch 178
Validation prc-auc = 0.455510
Epoch 179
Loss = 3.8566e-02, PNorm = 95.8629, GNorm = 0.1194, lr_0 = 7.4492e-04
Validation prc-auc = 0.442519
Epoch 180
Validation prc-auc = 0.438892
Epoch 181
Validation prc-auc = 0.434492
Epoch 182
Validation prc-auc = 0.433446
Epoch 183
Validation prc-auc = 0.436675
Epoch 184
Loss = 2.6134e-02, PNorm = 95.9451, GNorm = 1.7625, lr_0 = 7.3855e-04
Validation prc-auc = 0.439139
Epoch 185
Validation prc-auc = 0.434317
Epoch 186
Validation prc-auc = 0.420692
Epoch 187
Validation prc-auc = 0.429735
Epoch 188
Validation prc-auc = 0.429592
Epoch 189
Loss = 1.5447e-02, PNorm = 96.0369, GNorm = 0.4852, lr_0 = 7.3224e-04
Validation prc-auc = 0.427557
Epoch 190
Validation prc-auc = 0.429272
Epoch 191
Validation prc-auc = 0.428641
Epoch 192
Validation prc-auc = 0.426991
Epoch 193
Validation prc-auc = 0.425990
Epoch 194
Loss = 7.9193e-03, PNorm = 96.1323, GNorm = 0.3286, lr_0 = 7.2598e-04
Validation prc-auc = 0.429953
Epoch 195
Validation prc-auc = 0.430784
Epoch 196
Validation prc-auc = 0.449041
Epoch 197
Validation prc-auc = 0.449745
Epoch 198
Validation prc-auc = 0.454691
Epoch 199
Loss = 2.5388e-03, PNorm = 96.2198, GNorm = 0.0545, lr_0 = 7.1977e-04
Validation prc-auc = 0.458205
Epoch 200
Validation prc-auc = 0.438986
Epoch 201
Validation prc-auc = 0.429190
Epoch 202
Validation prc-auc = 0.423702
Epoch 203
Validation prc-auc = 0.428746
Epoch 204
Loss = 3.6089e-02, PNorm = 96.3380, GNorm = 0.6767, lr_0 = 7.1362e-04
Validation prc-auc = 0.423108
Epoch 205
Validation prc-auc = 0.432779
Epoch 206
Validation prc-auc = 0.407933
Epoch 207
Validation prc-auc = 0.406611
Epoch 208
Validation prc-auc = 0.401395
Epoch 209
Loss = 1.0056e-02, PNorm = 96.4829, GNorm = 0.7132, lr_0 = 7.0752e-04
Validation prc-auc = 0.406747
Epoch 210
Validation prc-auc = 0.418342
Epoch 211
Validation prc-auc = 0.427393
Epoch 212
Validation prc-auc = 0.433735
Epoch 213
Validation prc-auc = 0.442973
Epoch 214
Loss = 9.7111e-03, PNorm = 96.6129, GNorm = 0.0929, lr_0 = 7.0147e-04
Validation prc-auc = 0.458709
Epoch 215
Validation prc-auc = 0.464782
Epoch 216
Validation prc-auc = 0.464292
Epoch 217
Validation prc-auc = 0.457670
Epoch 218
Validation prc-auc = 0.460496
Epoch 219
Loss = 3.2640e-02, PNorm = 96.7670, GNorm = 0.2837, lr_0 = 6.9548e-04
Validation prc-auc = 0.441892
Epoch 220
Validation prc-auc = 0.444179
Epoch 221
Validation prc-auc = 0.443064
Epoch 222
Validation prc-auc = 0.443136
Epoch 223
Validation prc-auc = 0.440896
Epoch 224
Loss = 3.1552e-02, PNorm = 96.9032, GNorm = 0.0302, lr_0 = 6.8953e-04
Validation prc-auc = 0.430635
Epoch 225
Validation prc-auc = 0.432053
Epoch 226
Validation prc-auc = 0.397710
Epoch 227
Validation prc-auc = 0.437910
Epoch 228
Validation prc-auc = 0.451563
Epoch 229
Loss = 1.4617e-02, PNorm = 97.0343, GNorm = 1.0776, lr_0 = 6.8364e-04
Validation prc-auc = 0.450803
Epoch 230
Validation prc-auc = 0.452481
Epoch 231
Validation prc-auc = 0.456942
Epoch 232
Validation prc-auc = 0.457571
Epoch 233
Validation prc-auc = 0.457188
Epoch 234
Loss = 2.7235e-02, PNorm = 97.1663, GNorm = 1.0699, lr_0 = 6.7779e-04
Validation prc-auc = 0.455106
Epoch 235
Validation prc-auc = 0.461496
Epoch 236
Validation prc-auc = 0.464594
Epoch 237
Validation prc-auc = 0.457624
Epoch 238
Validation prc-auc = 0.449988
Epoch 239
Loss = 5.2762e-03, PNorm = 97.2618, GNorm = 0.4283, lr_0 = 6.7200e-04
Validation prc-auc = 0.446354
Epoch 240
Validation prc-auc = 0.438472
Epoch 241
Validation prc-auc = 0.437392
Epoch 242
Validation prc-auc = 0.430240
Epoch 243
Validation prc-auc = 0.431234
Epoch 244
Loss = 1.9619e-02, PNorm = 97.3400, GNorm = 1.6998, lr_0 = 6.6625e-04
Validation prc-auc = 0.437792
Epoch 245
Validation prc-auc = 0.436518
Epoch 246
Validation prc-auc = 0.437721
Epoch 247
Validation prc-auc = 0.423363
Epoch 248
Validation prc-auc = 0.423535
Epoch 249
Loss = 1.2185e-02, PNorm = 97.4052, GNorm = 0.6697, lr_0 = 6.6056e-04
Validation prc-auc = 0.420383
Epoch 250
Validation prc-auc = 0.424112
Epoch 251
Validation prc-auc = 0.426065
Epoch 252
Validation prc-auc = 0.419353
Epoch 253
Validation prc-auc = 0.420389
Epoch 254
Loss = 1.6059e-03, PNorm = 97.4702, GNorm = 0.0141, lr_0 = 6.5491e-04
Validation prc-auc = 0.434096
Epoch 255
Validation prc-auc = 0.432772
Epoch 256
Validation prc-auc = 0.434493
Epoch 257
Validation prc-auc = 0.433995
Epoch 258
Validation prc-auc = 0.426429
Epoch 259
Loss = 2.6584e-03, PNorm = 97.5203, GNorm = 0.0910, lr_0 = 6.4931e-04
Validation prc-auc = 0.425642
Epoch 260
Validation prc-auc = 0.432211
Epoch 261
Validation prc-auc = 0.430794
Epoch 262
Validation prc-auc = 0.431133
Epoch 263
Validation prc-auc = 0.430734
Epoch 264
Loss = 4.7283e-03, PNorm = 97.5618, GNorm = 0.4351, lr_0 = 6.4376e-04
Validation prc-auc = 0.431288
Epoch 265
Validation prc-auc = 0.423735
Epoch 266
Validation prc-auc = 0.432371
Epoch 267
Validation prc-auc = 0.419239
Epoch 268
Validation prc-auc = 0.418727
Epoch 269
Loss = 1.3339e-03, PNorm = 97.6013, GNorm = 0.0409, lr_0 = 6.3826e-04
Validation prc-auc = 0.419426
Epoch 270
Validation prc-auc = 0.419379
Epoch 271
Validation prc-auc = 0.417833
Epoch 272
Validation prc-auc = 0.418008
Epoch 273
Validation prc-auc = 0.413261
Epoch 274
Loss = 3.1545e-03, PNorm = 97.6340, GNorm = 0.1737, lr_0 = 6.3280e-04
Validation prc-auc = 0.413360
Epoch 275
Validation prc-auc = 0.419407
Epoch 276
Validation prc-auc = 0.419307
Epoch 277
Validation prc-auc = 0.418419
Epoch 278
Validation prc-auc = 0.418796
Epoch 279
Loss = 1.2229e-03, PNorm = 97.6530, GNorm = 0.0735, lr_0 = 6.2739e-04
Validation prc-auc = 0.418911
Epoch 280
Validation prc-auc = 0.428230
Epoch 281
Validation prc-auc = 0.428172
Epoch 282
Validation prc-auc = 0.425237
Epoch 283
Validation prc-auc = 0.429501
Epoch 284
Loss = 4.5133e-03, PNorm = 97.6680, GNorm = 0.2836, lr_0 = 6.2203e-04
Validation prc-auc = 0.433287
Epoch 285
Validation prc-auc = 0.431413
Epoch 286
Validation prc-auc = 0.430744
Epoch 287
Validation prc-auc = 0.431019
Epoch 288
Validation prc-auc = 0.431145
Epoch 289
Loss = 1.0324e-03, PNorm = 97.6807, GNorm = 0.0024, lr_0 = 6.1671e-04
Validation prc-auc = 0.431894
Epoch 290
Validation prc-auc = 0.432449
Epoch 291
Validation prc-auc = 0.431721
Epoch 292
Validation prc-auc = 0.431904
Epoch 293
Validation prc-auc = 0.439061
Epoch 294
Loss = 6.9168e-03, PNorm = 97.6960, GNorm = 0.4079, lr_0 = 6.1144e-04
Validation prc-auc = 0.447300
Epoch 295
Validation prc-auc = 0.447445
Epoch 296
Validation prc-auc = 0.446198
Epoch 297
Validation prc-auc = 0.446520
Epoch 298
Validation prc-auc = 0.446957
Epoch 299
Loss = 5.5389e-04, PNorm = 97.7199, GNorm = 0.0071, lr_0 = 6.0621e-04
Validation prc-auc = 0.445750
Model 0 best validation prc-auc = 0.485699 on epoch 83
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.208663
Ensemble test prc-auc = 0.208663
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.417140
Epoch 1
Validation prc-auc = 0.361958
Epoch 2
Validation prc-auc = 0.387800
Epoch 3
Validation prc-auc = 0.300782
Epoch 4
Loss = 5.7426e-01, PNorm = 87.5282, GNorm = 1.6274, lr_0 = 6.5000e-04
Validation prc-auc = 0.309369
Epoch 5
Validation prc-auc = 0.401315
Epoch 6
Validation prc-auc = 0.424247
Epoch 7
Validation prc-auc = 0.367953
Epoch 8
Validation prc-auc = 0.333095
Epoch 9
Loss = 4.3376e-01, PNorm = 87.8611, GNorm = 1.1699, lr_0 = 9.9743e-04
Validation prc-auc = 0.411416
Epoch 10
Validation prc-auc = 0.405098
Epoch 11
Validation prc-auc = 0.334553
Epoch 12
Validation prc-auc = 0.224478
Epoch 13
Validation prc-auc = 0.220262
Epoch 14
Loss = 3.2895e-01, PNorm = 88.2263, GNorm = 1.3673, lr_0 = 9.8890e-04
Validation prc-auc = 0.304726
Epoch 15
Validation prc-auc = 0.444088
Epoch 16
Validation prc-auc = 0.395855
Epoch 17
Validation prc-auc = 0.330267
Epoch 18
Validation prc-auc = 0.317535
Epoch 19
Loss = 2.9843e-01, PNorm = 88.4972, GNorm = 1.0106, lr_0 = 9.8045e-04
Validation prc-auc = 0.329056
Epoch 20
Validation prc-auc = 0.349797
Epoch 21
Validation prc-auc = 0.195643
Epoch 22
Validation prc-auc = 0.183077
Epoch 23
Validation prc-auc = 0.393936
Epoch 24
Loss = 3.2132e-01, PNorm = 88.7283, GNorm = 2.7045, lr_0 = 9.7207e-04
Validation prc-auc = 0.438623
Epoch 25
Validation prc-auc = 0.407981
Epoch 26
Validation prc-auc = 0.244544
Epoch 27
Validation prc-auc = 0.231576
Epoch 28
Validation prc-auc = 0.277548
Epoch 29
Loss = 2.1079e-01, PNorm = 88.9412, GNorm = 1.7030, lr_0 = 9.6376e-04
Validation prc-auc = 0.391745
Epoch 30
Validation prc-auc = 0.411553
Epoch 31
Validation prc-auc = 0.401958
Epoch 32
Validation prc-auc = 0.302568
Epoch 33
Validation prc-auc = 0.342746
Epoch 34
Loss = 2.5944e-01, PNorm = 89.1329, GNorm = 1.9810, lr_0 = 9.5552e-04
Validation prc-auc = 0.403542
Epoch 35
Validation prc-auc = 0.387372
Epoch 36
Validation prc-auc = 0.413032
Epoch 37
Validation prc-auc = 0.410060
Epoch 38
Validation prc-auc = 0.264293
Epoch 39
Loss = 3.6827e-01, PNorm = 89.3444, GNorm = 1.8152, lr_0 = 9.4735e-04
Validation prc-auc = 0.328508
Epoch 40
Validation prc-auc = 0.415854
Epoch 41
Validation prc-auc = 0.419574
Epoch 42
Validation prc-auc = 0.280121
Epoch 43
Validation prc-auc = 0.378350
Epoch 44
Loss = 2.4889e-01, PNorm = 89.5614, GNorm = 1.4404, lr_0 = 9.3925e-04
Validation prc-auc = 0.424026
Epoch 45
Validation prc-auc = 0.440040
Epoch 46
Validation prc-auc = 0.434883
Epoch 47
Validation prc-auc = 0.430711
Epoch 48
Validation prc-auc = 0.425960
Epoch 49
Loss = 1.3379e-01, PNorm = 89.7906, GNorm = 0.7518, lr_0 = 9.3122e-04
Validation prc-auc = 0.386787
Epoch 50
Validation prc-auc = 0.377109
Epoch 51
Validation prc-auc = 0.391445
Epoch 52
Validation prc-auc = 0.415187
Epoch 53
Validation prc-auc = 0.435488
Epoch 54
Loss = 1.6290e-01, PNorm = 90.0032, GNorm = 0.9646, lr_0 = 9.2326e-04
Validation prc-auc = 0.444859
Epoch 55
Validation prc-auc = 0.446069
Epoch 56
Validation prc-auc = 0.449606
Epoch 57
Validation prc-auc = 0.410217
Epoch 58
Validation prc-auc = 0.394327
Epoch 59
Loss = 1.4357e-01, PNorm = 90.2202, GNorm = 1.1382, lr_0 = 9.1537e-04
Validation prc-auc = 0.428181
Epoch 60
Validation prc-auc = 0.457508
Epoch 61
Validation prc-auc = 0.454725
Epoch 62
Validation prc-auc = 0.438094
Epoch 63
Validation prc-auc = 0.413975
Epoch 64
Loss = 2.4623e-01, PNorm = 90.4661, GNorm = 1.3880, lr_0 = 9.0754e-04
Validation prc-auc = 0.306413
Epoch 65
Validation prc-auc = 0.265867
Epoch 66
Validation prc-auc = 0.450975
Epoch 67
Validation prc-auc = 0.438370
Epoch 68
Validation prc-auc = 0.466600
Epoch 69
Loss = 1.7580e-01, PNorm = 90.7507, GNorm = 1.7710, lr_0 = 8.9978e-04
Validation prc-auc = 0.449782
Epoch 70
Validation prc-auc = 0.401458
Epoch 71
Validation prc-auc = 0.445361
Epoch 72
Validation prc-auc = 0.450468
Epoch 73
Validation prc-auc = 0.414953
Epoch 74
Loss = 1.5062e-01, PNorm = 91.0858, GNorm = 2.4687, lr_0 = 8.9209e-04
Validation prc-auc = 0.416123
Epoch 75
Validation prc-auc = 0.453903
Epoch 76
Validation prc-auc = 0.422909
Epoch 77
Validation prc-auc = 0.402362
Epoch 78
Validation prc-auc = 0.342812
Epoch 79
Loss = 1.8211e-01, PNorm = 91.4121, GNorm = 1.9005, lr_0 = 8.8447e-04
Validation prc-auc = 0.308887
Epoch 80
Validation prc-auc = 0.375723
Epoch 81
Validation prc-auc = 0.406563
Epoch 82
Validation prc-auc = 0.420719
Epoch 83
Validation prc-auc = 0.428756
Epoch 84
Loss = 1.2189e-01, PNorm = 91.7455, GNorm = 0.5879, lr_0 = 8.7691e-04
Validation prc-auc = 0.423498
Epoch 85
Validation prc-auc = 0.416329
Epoch 86
Validation prc-auc = 0.419539
Epoch 87
Validation prc-auc = 0.360348
Epoch 88
Validation prc-auc = 0.387305
Epoch 89
Loss = 7.4105e-02, PNorm = 91.9942, GNorm = 0.5748, lr_0 = 8.6941e-04
Validation prc-auc = 0.432944
Epoch 90
Validation prc-auc = 0.437709
Epoch 91
Validation prc-auc = 0.444306
Epoch 92
Validation prc-auc = 0.435510
Epoch 93
Validation prc-auc = 0.432036
Epoch 94
Loss = 2.5803e-02, PNorm = 92.2194, GNorm = 0.2596, lr_0 = 8.6198e-04
Validation prc-auc = 0.411180
Epoch 95
Validation prc-auc = 0.412339
Epoch 96
Validation prc-auc = 0.426122
Epoch 97
Validation prc-auc = 0.419092
Epoch 98
Validation prc-auc = 0.405961
Epoch 99
Loss = 1.2640e-01, PNorm = 92.4479, GNorm = 1.6062, lr_0 = 8.5461e-04
Validation prc-auc = 0.386637
Epoch 100
Validation prc-auc = 0.366762
Epoch 101
Validation prc-auc = 0.440471
Epoch 102
Validation prc-auc = 0.436802
Epoch 103
Validation prc-auc = 0.450383
Epoch 104
Loss = 5.2025e-02, PNorm = 92.6969, GNorm = 0.6953, lr_0 = 8.4730e-04
Validation prc-auc = 0.435181
Epoch 105
Validation prc-auc = 0.428768
Epoch 106
Validation prc-auc = 0.445100
Epoch 107
Validation prc-auc = 0.441315
Epoch 108
Validation prc-auc = 0.441089
Epoch 109
Loss = 5.5837e-02, PNorm = 92.9366, GNorm = 0.5485, lr_0 = 8.4006e-04
Validation prc-auc = 0.449626
Epoch 110
Validation prc-auc = 0.435515
Epoch 111
Validation prc-auc = 0.447421
Epoch 112
Validation prc-auc = 0.420420
Epoch 113
Validation prc-auc = 0.430953
Epoch 114
Loss = 1.0799e-01, PNorm = 93.1663, GNorm = 0.6536, lr_0 = 8.3288e-04
Validation prc-auc = 0.415433
Epoch 115
Validation prc-auc = 0.416180
Epoch 116
Validation prc-auc = 0.421352
Epoch 117
Validation prc-auc = 0.432354
Epoch 118
Validation prc-auc = 0.424897
Epoch 119
Loss = 2.3889e-02, PNorm = 93.4201, GNorm = 0.5709, lr_0 = 8.2576e-04
Validation prc-auc = 0.445203
Epoch 120
Validation prc-auc = 0.445953
Epoch 121
Validation prc-auc = 0.446310
Epoch 122
Validation prc-auc = 0.430029
Epoch 123
Validation prc-auc = 0.421010
Epoch 124
Loss = 6.5370e-02, PNorm = 93.6379, GNorm = 1.3540, lr_0 = 8.1870e-04
Validation prc-auc = 0.438989
Epoch 125
Validation prc-auc = 0.447093
Epoch 126
Validation prc-auc = 0.446753
Epoch 127
Validation prc-auc = 0.449263
Epoch 128
Validation prc-auc = 0.438370
Epoch 129
Loss = 2.5550e-02, PNorm = 93.8404, GNorm = 0.2607, lr_0 = 8.1170e-04
Validation prc-auc = 0.425649
Epoch 130
Validation prc-auc = 0.426348
Epoch 131
Validation prc-auc = 0.435300
Epoch 132
Validation prc-auc = 0.424107
Epoch 133
Validation prc-auc = 0.428003
Epoch 134
Loss = 7.0330e-02, PNorm = 94.0227, GNorm = 1.6176, lr_0 = 8.0476e-04
Validation prc-auc = 0.430016
Epoch 135
Validation prc-auc = 0.419811
Epoch 136
Validation prc-auc = 0.426696
Epoch 137
Validation prc-auc = 0.435715
Epoch 138
Validation prc-auc = 0.432525
Epoch 139
Loss = 9.6485e-02, PNorm = 94.1756, GNorm = 1.4552, lr_0 = 7.9788e-04
Validation prc-auc = 0.447169
Epoch 140
Validation prc-auc = 0.455961
Epoch 141
Validation prc-auc = 0.451076
Epoch 142
Validation prc-auc = 0.457029
Epoch 143
Validation prc-auc = 0.460585
Epoch 144
Loss = 5.4096e-02, PNorm = 94.3323, GNorm = 2.5794, lr_0 = 7.9106e-04
Validation prc-auc = 0.449429
Epoch 145
Validation prc-auc = 0.443831
Epoch 146
Validation prc-auc = 0.445604
Epoch 147
Validation prc-auc = 0.438827
Epoch 148
Validation prc-auc = 0.429713
Epoch 149
Loss = 1.3550e-01, PNorm = 94.5290, GNorm = 0.6260, lr_0 = 7.8430e-04
Validation prc-auc = 0.418858
Epoch 150
Validation prc-auc = 0.440084
Epoch 151
Validation prc-auc = 0.450278
Epoch 152
Validation prc-auc = 0.451350
Epoch 153
Validation prc-auc = 0.448242
Epoch 154
Loss = 1.8377e-02, PNorm = 94.7197, GNorm = 0.4570, lr_0 = 7.7759e-04
Validation prc-auc = 0.456159
Epoch 155
Validation prc-auc = 0.422200
Epoch 156
Validation prc-auc = 0.424029
Epoch 157
Validation prc-auc = 0.424714
Epoch 158
Validation prc-auc = 0.424753
Epoch 159
Loss = 3.4250e-02, PNorm = 94.8492, GNorm = 1.6143, lr_0 = 7.7095e-04
Validation prc-auc = 0.411210
Epoch 160
Validation prc-auc = 0.370185
Epoch 161
Validation prc-auc = 0.370150
Epoch 162
Validation prc-auc = 0.387450
Epoch 163
Validation prc-auc = 0.390577
Epoch 164
Loss = 1.4643e-02, PNorm = 94.9639, GNorm = 0.3279, lr_0 = 7.6436e-04
Validation prc-auc = 0.397674
Epoch 165
Validation prc-auc = 0.410338
Epoch 166
Validation prc-auc = 0.428907
Epoch 167
Validation prc-auc = 0.453313
Epoch 168
Validation prc-auc = 0.460441
Epoch 169
Loss = 3.7302e-02, PNorm = 95.1010, GNorm = 0.1342, lr_0 = 7.5782e-04
Validation prc-auc = 0.460181
Epoch 170
Validation prc-auc = 0.454117
Epoch 171
Validation prc-auc = 0.446764
Epoch 172
Validation prc-auc = 0.441415
Epoch 173
Validation prc-auc = 0.429278
Epoch 174
Loss = 5.4835e-02, PNorm = 95.2432, GNorm = 0.8139, lr_0 = 7.5134e-04
Validation prc-auc = 0.426673
Epoch 175
Validation prc-auc = 0.411886
Epoch 176
Validation prc-auc = 0.412941
Epoch 177
Validation prc-auc = 0.409490
Epoch 178
Validation prc-auc = 0.407178
Epoch 179
Loss = 3.8957e-02, PNorm = 95.3619, GNorm = 1.1196, lr_0 = 7.4492e-04
Validation prc-auc = 0.407024
Epoch 180
Validation prc-auc = 0.402807
Epoch 181
Validation prc-auc = 0.402274
Epoch 182
Validation prc-auc = 0.405391
Epoch 183
Validation prc-auc = 0.403483
Epoch 184
Loss = 1.2778e-02, PNorm = 95.4866, GNorm = 0.4562, lr_0 = 7.3855e-04
Validation prc-auc = 0.398086
Epoch 185
Validation prc-auc = 0.393007
Epoch 186
Validation prc-auc = 0.397192
Epoch 187
Validation prc-auc = 0.386654
Epoch 188
Validation prc-auc = 0.411282
Epoch 189
Loss = 8.1843e-03, PNorm = 95.6057, GNorm = 0.0934, lr_0 = 7.3224e-04
Validation prc-auc = 0.414968
Epoch 190
Validation prc-auc = 0.414224
Epoch 191
Validation prc-auc = 0.413908
Epoch 192
Validation prc-auc = 0.418461
Epoch 193
Validation prc-auc = 0.416951
Epoch 194
Loss = 1.4833e-02, PNorm = 95.6944, GNorm = 0.1407, lr_0 = 7.2598e-04
Validation prc-auc = 0.415620
Epoch 195
Validation prc-auc = 0.420586
Epoch 196
Validation prc-auc = 0.423811
Epoch 197
Validation prc-auc = 0.412442
Epoch 198
Validation prc-auc = 0.414813
Epoch 199
Loss = 6.7998e-03, PNorm = 95.7659, GNorm = 0.5240, lr_0 = 7.1977e-04
Validation prc-auc = 0.432260
Epoch 200
Validation prc-auc = 0.430593
Epoch 201
Validation prc-auc = 0.436779
Epoch 202
Validation prc-auc = 0.436646
Epoch 203
Validation prc-auc = 0.432722
Epoch 204
Loss = 1.8404e-03, PNorm = 95.8368, GNorm = 0.0590, lr_0 = 7.1362e-04
Validation prc-auc = 0.428541
Epoch 205
Validation prc-auc = 0.428722
Epoch 206
Validation prc-auc = 0.423335
Epoch 207
Validation prc-auc = 0.411534
Epoch 208
Validation prc-auc = 0.406874
Epoch 209
Loss = 2.1791e-03, PNorm = 95.8834, GNorm = 0.0181, lr_0 = 7.0752e-04
Validation prc-auc = 0.419977
Epoch 210
Validation prc-auc = 0.416317
Epoch 211
Validation prc-auc = 0.409035
Epoch 212
Validation prc-auc = 0.401800
Epoch 213
Validation prc-auc = 0.400504
Epoch 214
Loss = 2.3672e-03, PNorm = 95.9279, GNorm = 0.0592, lr_0 = 7.0147e-04
Validation prc-auc = 0.401136
Epoch 215
Validation prc-auc = 0.399045
Epoch 216
Validation prc-auc = 0.399675
Epoch 217
Validation prc-auc = 0.400557
Epoch 218
Validation prc-auc = 0.412970
Epoch 219
Loss = 2.3854e-03, PNorm = 95.9711, GNorm = 0.1827, lr_0 = 6.9548e-04
Validation prc-auc = 0.414744
Epoch 220
Validation prc-auc = 0.414649
Epoch 221
Validation prc-auc = 0.414953
Epoch 222
Validation prc-auc = 0.414954
Epoch 223
Validation prc-auc = 0.414774
Epoch 224
Loss = 2.7659e-03, PNorm = 95.9985, GNorm = 0.0624, lr_0 = 6.8953e-04
Validation prc-auc = 0.413522
Epoch 225
Validation prc-auc = 0.410891
Epoch 226
Validation prc-auc = 0.411168
Epoch 227
Validation prc-auc = 0.421522
Epoch 228
Validation prc-auc = 0.422051
Epoch 229
Loss = 2.8397e-02, PNorm = 96.0201, GNorm = 1.1610, lr_0 = 6.8364e-04
Validation prc-auc = 0.423201
Epoch 230
Validation prc-auc = 0.422744
Epoch 231
Validation prc-auc = 0.425063
Epoch 232
Validation prc-auc = 0.425150
Epoch 233
Validation prc-auc = 0.424191
Epoch 234
Loss = 2.5913e-03, PNorm = 96.0632, GNorm = 0.0591, lr_0 = 6.7779e-04
Validation prc-auc = 0.432355
Epoch 235
Validation prc-auc = 0.434030
Epoch 236
Validation prc-auc = 0.436442
Epoch 237
Validation prc-auc = 0.437796
Epoch 238
Validation prc-auc = 0.438406
Epoch 239
Loss = 4.2207e-04, PNorm = 96.1176, GNorm = 0.0593, lr_0 = 6.7200e-04
Validation prc-auc = 0.442943
Epoch 240
Validation prc-auc = 0.442689
Epoch 241
Validation prc-auc = 0.453374
Epoch 242
Validation prc-auc = 0.452215
Epoch 243
Validation prc-auc = 0.435163
Epoch 244
Loss = 1.8371e-02, PNorm = 96.1773, GNorm = 1.1020, lr_0 = 6.6625e-04
Validation prc-auc = 0.433300
Epoch 245
Validation prc-auc = 0.429790
Epoch 246
Validation prc-auc = 0.428968
Epoch 247
Validation prc-auc = 0.430677
Epoch 248
Validation prc-auc = 0.435158
Epoch 249
Loss = 4.5608e-03, PNorm = 96.2605, GNorm = 0.4061, lr_0 = 6.6056e-04
Validation prc-auc = 0.445980
Epoch 250
Validation prc-auc = 0.433837
Epoch 251
Validation prc-auc = 0.428183
Epoch 252
Validation prc-auc = 0.421520
Epoch 253
Validation prc-auc = 0.419559
Epoch 254
Loss = 9.2313e-03, PNorm = 96.3414, GNorm = 0.0255, lr_0 = 6.5491e-04
Validation prc-auc = 0.417092
Epoch 255
Validation prc-auc = 0.417992
Epoch 256
Validation prc-auc = 0.411962
Epoch 257
Validation prc-auc = 0.406484
Epoch 258
Validation prc-auc = 0.397154
Epoch 259
Loss = 1.9324e-02, PNorm = 96.4416, GNorm = 0.9543, lr_0 = 6.4931e-04
Validation prc-auc = 0.392298
Epoch 260
Validation prc-auc = 0.392948
Epoch 261
Validation prc-auc = 0.375776
Epoch 262
Validation prc-auc = 0.383132
Epoch 263
Validation prc-auc = 0.386357
Epoch 264
Loss = 6.1689e-03, PNorm = 96.5259, GNorm = 0.0465, lr_0 = 6.4376e-04
Validation prc-auc = 0.377211
Epoch 265
Validation prc-auc = 0.374431
Epoch 266
Validation prc-auc = 0.374427
Epoch 267
Validation prc-auc = 0.401896
Epoch 268
Validation prc-auc = 0.417085
Epoch 269
Loss = 8.3199e-03, PNorm = 96.5984, GNorm = 0.0310, lr_0 = 6.3826e-04
Validation prc-auc = 0.427995
Epoch 270
Validation prc-auc = 0.424188
Epoch 271
Validation prc-auc = 0.424453
Epoch 272
Validation prc-auc = 0.422540
Epoch 273
Validation prc-auc = 0.414268
Epoch 274
Loss = 2.4998e-03, PNorm = 96.6796, GNorm = 0.0733, lr_0 = 6.3280e-04
Validation prc-auc = 0.411025
Epoch 275
Validation prc-auc = 0.408465
Epoch 276
Validation prc-auc = 0.404615
Epoch 277
Validation prc-auc = 0.370630
Epoch 278
Validation prc-auc = 0.389073
Epoch 279
Loss = 6.3124e-02, PNorm = 96.7655, GNorm = 2.4554, lr_0 = 6.2739e-04
Validation prc-auc = 0.402500
Epoch 280
Validation prc-auc = 0.430557
Epoch 281
Validation prc-auc = 0.415519
Epoch 282
Validation prc-auc = 0.421414
Epoch 283
Validation prc-auc = 0.429479
Epoch 284
Loss = 1.8650e-03, PNorm = 96.8805, GNorm = 0.1761, lr_0 = 6.2203e-04
Validation prc-auc = 0.410007
Epoch 285
Validation prc-auc = 0.404690
Epoch 286
Validation prc-auc = 0.427880
Epoch 287
Validation prc-auc = 0.409358
Epoch 288
Validation prc-auc = 0.408854
Epoch 289
Loss = 3.4157e-03, PNorm = 97.0018, GNorm = 0.1945, lr_0 = 6.1671e-04
Validation prc-auc = 0.393334
Epoch 290
Validation prc-auc = 0.392209
Epoch 291
Validation prc-auc = 0.383726
Epoch 292
Validation prc-auc = 0.376933
Epoch 293
Validation prc-auc = 0.376433
Epoch 294
Loss = 1.2739e-03, PNorm = 97.1137, GNorm = 0.0459, lr_0 = 6.1144e-04
Validation prc-auc = 0.378681
Epoch 295
Validation prc-auc = 0.384073
Epoch 296
Validation prc-auc = 0.402107
Epoch 297
Validation prc-auc = 0.422976
Epoch 298
Validation prc-auc = 0.429092
Epoch 299
Loss = 1.2858e-02, PNorm = 97.2142, GNorm = 0.8068, lr_0 = 6.0621e-04
Validation prc-auc = 0.433400
Model 0 best validation prc-auc = 0.466600 on epoch 68
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.195464
Ensemble test prc-auc = 0.195464
10-fold cross validation
	Seed 0 ==> test prc-auc = 0.253857
	Seed 1 ==> test prc-auc = 0.320228
	Seed 2 ==> test prc-auc = 0.215171
	Seed 3 ==> test prc-auc = 0.324470
	Seed 4 ==> test prc-auc = 0.322710
	Seed 5 ==> test prc-auc = 0.254939
	Seed 6 ==> test prc-auc = 0.315661
	Seed 7 ==> test prc-auc = 0.244857
	Seed 8 ==> test prc-auc = 0.208663
	Seed 9 ==> test prc-auc = 0.195464
Overall test prc-auc = 0.265602 +/- 0.048583
Elapsed time = 0:07:27
Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/with_whim/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/with_whim/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=0),
 'dropout': 0.25,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/train_whim_mean.npz',
                   '/home/saxelrod/chemprop_cov_2/features/whim/train_whim_std.npz'],
 'features_scaling': True,
 'features_size': None,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'grad_clip': None,
 'hidden_size': 1800,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'metrics': ['prc-auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': False,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/with_whim',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/test_whim_mean.npz',
                                 '/home/saxelrod/chemprop_cov_2/features/whim/test_whim_std.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/val_whim_mean.npz',
                                '/home/saxelrod/chemprop_cov_2/features/whim/val_whim_std.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/with_whim/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/with_whim/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=0),
 'dropout': 0.25,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/train_whim_mean.npz',
                   '/home/saxelrod/chemprop_cov_2/features/whim/train_whim_std.npz'],
 'features_scaling': True,
 'features_size': None,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'grad_clip': None,
 'hidden_size': 1800,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'metrics': ['prc-auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': False,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/with_whim',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/test_whim_mean.npz',
                                 '/home/saxelrod/chemprop_cov_2/features/whim/test_whim_std.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/whim/val_whim_mean.npz',
                                '/home/saxelrod/chemprop_cov_2/features/whim/val_whim_std.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.344032
Epoch 1
Validation prc-auc = 0.370081
Epoch 2
Validation prc-auc = 0.449001
Epoch 3
Validation prc-auc = 0.367189
Epoch 4
Loss = 5.5639e-01, PNorm = 87.5388, GNorm = 3.6398, lr_0 = 6.5000e-04
Validation prc-auc = 0.364293
Epoch 5
Validation prc-auc = 0.272900
Epoch 6
Validation prc-auc = 0.375091
Epoch 7
Validation prc-auc = 0.436418
Epoch 8
Validation prc-auc = 0.181976
Epoch 9
Loss = 6.0173e-01, PNorm = 87.8840, GNorm = 3.2223, lr_0 = 9.9743e-04
Validation prc-auc = 0.201774
Epoch 10
Validation prc-auc = 0.451486
Epoch 11
Validation prc-auc = 0.448606
Epoch 12
Validation prc-auc = 0.312307
Epoch 13
Validation prc-auc = 0.201565
Epoch 14
Loss = 5.1086e-01, PNorm = 88.2679, GNorm = 1.6762, lr_0 = 9.8890e-04
Validation prc-auc = 0.414950
Epoch 15
Validation prc-auc = 0.381266
Epoch 16
Validation prc-auc = 0.379147
Epoch 17
Validation prc-auc = 0.367681
Epoch 18
Validation prc-auc = 0.377430
Epoch 19
Loss = 2.7039e-01, PNorm = 88.5625, GNorm = 1.1058, lr_0 = 9.8045e-04
Validation prc-auc = 0.407147
Epoch 20
Validation prc-auc = 0.433739
Epoch 21
Validation prc-auc = 0.387721
Epoch 22
Validation prc-auc = 0.387218
Epoch 23
Validation prc-auc = 0.411899
Epoch 24
Loss = 2.0175e-01, PNorm = 88.7860, GNorm = 1.0824, lr_0 = 9.7207e-04
Validation prc-auc = 0.409816
Epoch 25
Validation prc-auc = 0.418001
Epoch 26
Validation prc-auc = 0.405500
Epoch 27
Validation prc-auc = 0.377260
Epoch 28
Validation prc-auc = 0.393469
Epoch 29
Loss = 2.6747e-01, PNorm = 88.9744, GNorm = 1.9238, lr_0 = 9.6376e-04
Validation prc-auc = 0.374732
Epoch 30
Validation prc-auc = 0.396081
Epoch 31
Validation prc-auc = 0.402644
Epoch 32
Validation prc-auc = 0.410021
Epoch 33
Validation prc-auc = 0.421860
Epoch 34
Loss = 2.2050e-01, PNorm = 89.1732, GNorm = 2.5255, lr_0 = 9.5552e-04
Validation prc-auc = 0.444874
Epoch 35
Validation prc-auc = 0.439026
Epoch 36
Validation prc-auc = 0.379474
Epoch 37
Validation prc-auc = 0.230645
Epoch 38
Validation prc-auc = 0.335320
Epoch 39
Loss = 2.5741e-01, PNorm = 89.4055, GNorm = 1.9334, lr_0 = 9.4735e-04
Validation prc-auc = 0.408657
Epoch 40
Validation prc-auc = 0.395392
Epoch 41
Validation prc-auc = 0.329612
Epoch 42
Validation prc-auc = 0.343811
Epoch 43
Validation prc-auc = 0.428889
Epoch 44
Loss = 2.2029e-01, PNorm = 89.6387, GNorm = 1.6285, lr_0 = 9.3925e-04
Validation prc-auc = 0.434581
Epoch 45
Validation prc-auc = 0.452023
Epoch 46
Validation prc-auc = 0.457713
Epoch 47
Validation prc-auc = 0.466346
Epoch 48
Validation prc-auc = 0.456176
Epoch 49
Loss = 1.4514e-01, PNorm = 89.8957, GNorm = 0.9208, lr_0 = 9.3122e-04
Validation prc-auc = 0.451238
Epoch 50
Validation prc-auc = 0.449543
Epoch 51
Validation prc-auc = 0.386671
Epoch 52
Validation prc-auc = 0.381045
Epoch 53
Validation prc-auc = 0.428997
Epoch 54
Loss = 2.7166e-01, PNorm = 90.1701, GNorm = 2.1145, lr_0 = 9.2326e-04
Validation prc-auc = 0.439720
Epoch 55
Validation prc-auc = 0.388924
Epoch 56
Validation prc-auc = 0.340996
Epoch 57
Validation prc-auc = 0.442894
Epoch 58
Validation prc-auc = 0.423212
Epoch 59
Loss = 1.9060e-01, PNorm = 90.4601, GNorm = 1.7253, lr_0 = 9.1537e-04
Validation prc-auc = 0.414663
Epoch 60
Validation prc-auc = 0.437408
Epoch 61
Validation prc-auc = 0.415096
Epoch 62
Validation prc-auc = 0.408745
Epoch 63
Validation prc-auc = 0.425881
Epoch 64
Loss = 1.1534e-01, PNorm = 90.7538, GNorm = 2.5757, lr_0 = 9.0754e-04
Validation prc-auc = 0.430720
Epoch 65
Validation prc-auc = 0.435945
Epoch 66
Validation prc-auc = 0.438941
Epoch 67
Validation prc-auc = 0.434390
Epoch 68
Validation prc-auc = 0.425816
Epoch 69
Loss = 8.6286e-02, PNorm = 91.0123, GNorm = 1.5141, lr_0 = 8.9978e-04
Validation prc-auc = 0.428997
Epoch 70
Validation prc-auc = 0.422178
Epoch 71
Validation prc-auc = 0.427248
Epoch 72
Validation prc-auc = 0.424575
Epoch 73
Validation prc-auc = 0.432929
Epoch 74
Loss = 8.4085e-02, PNorm = 91.2715, GNorm = 0.4476, lr_0 = 8.9209e-04
Validation prc-auc = 0.442952
Epoch 75
Validation prc-auc = 0.434066
Epoch 76
Validation prc-auc = 0.429491
Epoch 77
Validation prc-auc = 0.435671
Epoch 78
Validation prc-auc = 0.277468
Epoch 79
Loss = 2.6304e-01, PNorm = 91.5420, GNorm = 0.4269, lr_0 = 8.8447e-04
Validation prc-auc = 0.355557
Epoch 80
Validation prc-auc = 0.402430
Epoch 81
Validation prc-auc = 0.416577
Epoch 82
Validation prc-auc = 0.428603
Epoch 83
Validation prc-auc = 0.430011
Epoch 84
Loss = 3.2294e-02, PNorm = 91.8748, GNorm = 0.5559, lr_0 = 8.7691e-04
Validation prc-auc = 0.427897
Epoch 85
Validation prc-auc = 0.409554
Epoch 86
Validation prc-auc = 0.413582
Epoch 87
Validation prc-auc = 0.439359
Epoch 88
Validation prc-auc = 0.437639
Epoch 89
Loss = 4.4130e-02, PNorm = 92.1427, GNorm = 0.8959, lr_0 = 8.6941e-04
Validation prc-auc = 0.439012
Epoch 90
Validation prc-auc = 0.439683
Epoch 91
Validation prc-auc = 0.433961
Epoch 92
Validation prc-auc = 0.417989
Epoch 93
Validation prc-auc = 0.427063
Epoch 94
Loss = 8.8655e-02, PNorm = 92.3736, GNorm = 1.4913, lr_0 = 8.6198e-04
Validation prc-auc = 0.426994
Epoch 95
Validation prc-auc = 0.446954
Epoch 96
Validation prc-auc = 0.421075
Epoch 97
Validation prc-auc = 0.416311
Epoch 98
Validation prc-auc = 0.389062
Epoch 99
Loss = 7.9109e-02, PNorm = 92.6326, GNorm = 0.6071, lr_0 = 8.5461e-04
Validation prc-auc = 0.401700
Epoch 100
Validation prc-auc = 0.428701
Epoch 101
Validation prc-auc = 0.436339
Epoch 102
Validation prc-auc = 0.423489
Epoch 103
Validation prc-auc = 0.438915
Epoch 104
Loss = 3.0024e-02, PNorm = 92.8305, GNorm = 0.5560, lr_0 = 8.4730e-04
Validation prc-auc = 0.446173
Epoch 105
Validation prc-auc = 0.420752
Epoch 106
Validation prc-auc = 0.420294
Epoch 107
Validation prc-auc = 0.443926
Epoch 108
Validation prc-auc = 0.454956
Epoch 109
Loss = 5.1735e-02, PNorm = 93.0339, GNorm = 1.0496, lr_0 = 8.4006e-04
Validation prc-auc = 0.447133
Epoch 110
Validation prc-auc = 0.417778
Epoch 111
Validation prc-auc = 0.421909
Epoch 112
Validation prc-auc = 0.438369
Epoch 113
Validation prc-auc = 0.437625
Epoch 114
Loss = 4.3535e-02, PNorm = 93.2592, GNorm = 1.4929, lr_0 = 8.3288e-04
Validation prc-auc = 0.433268
Epoch 115
Validation prc-auc = 0.423740
Epoch 116
Validation prc-auc = 0.398119
Epoch 117
Validation prc-auc = 0.394202
Epoch 118
Validation prc-auc = 0.410578
Epoch 119
Loss = 3.2450e-02, PNorm = 93.5180, GNorm = 1.3669, lr_0 = 8.2576e-04
Validation prc-auc = 0.432554
Epoch 120
Validation prc-auc = 0.434887
Epoch 121
Validation prc-auc = 0.424807
Epoch 122
Validation prc-auc = 0.428857
Epoch 123
Validation prc-auc = 0.430937
Epoch 124
Loss = 1.7406e-01, PNorm = 93.7800, GNorm = 0.5264, lr_0 = 8.1870e-04
Validation prc-auc = 0.422067
Epoch 125
Validation prc-auc = 0.411398
Epoch 126
Validation prc-auc = 0.331280
Epoch 127
Validation prc-auc = 0.269805
Epoch 128
Validation prc-auc = 0.315200
Epoch 129
Loss = 6.2494e-02, PNorm = 94.0509, GNorm = 1.1113, lr_0 = 8.1170e-04
Validation prc-auc = 0.447756
Epoch 130
Validation prc-auc = 0.447963
Epoch 131
Validation prc-auc = 0.462135
Epoch 132
Validation prc-auc = 0.447604
Epoch 133
Validation prc-auc = 0.443785
Epoch 134
Loss = 2.5817e-02, PNorm = 94.3241, GNorm = 0.7628, lr_0 = 8.0476e-04
Validation prc-auc = 0.444297
Epoch 135
Validation prc-auc = 0.418349
Epoch 136
Validation prc-auc = 0.422797
Epoch 137
Validation prc-auc = 0.429844
Epoch 138
Validation prc-auc = 0.414550
Epoch 139
Loss = 4.0557e-02, PNorm = 94.5242, GNorm = 0.9164, lr_0 = 7.9788e-04
Validation prc-auc = 0.415404
Epoch 140
Validation prc-auc = 0.433759
Epoch 141
Validation prc-auc = 0.436237
Epoch 142
Validation prc-auc = 0.438637
Epoch 143
Validation prc-auc = 0.449258
Epoch 144
Loss = 1.9007e-02, PNorm = 94.6817, GNorm = 0.1759, lr_0 = 7.9106e-04
Validation prc-auc = 0.449377
Epoch 145
Validation prc-auc = 0.448559
Epoch 146
Validation prc-auc = 0.448154
Epoch 147
Validation prc-auc = 0.435598
Epoch 148
Validation prc-auc = 0.440601
Epoch 149
Loss = 1.1710e-01, PNorm = 94.8340, GNorm = 1.0088, lr_0 = 7.8430e-04
Validation prc-auc = 0.446654
Epoch 150
Validation prc-auc = 0.438296
Epoch 151
Validation prc-auc = 0.439288
Epoch 152
Validation prc-auc = 0.442337
Epoch 153
Validation prc-auc = 0.434571
Epoch 154
Loss = 1.3031e-02, PNorm = 94.9879, GNorm = 0.0701, lr_0 = 7.7759e-04
Validation prc-auc = 0.442115
Epoch 155
Validation prc-auc = 0.431921
Epoch 156
Validation prc-auc = 0.421128
Epoch 157
Validation prc-auc = 0.411790
Epoch 158
Validation prc-auc = 0.421892
Epoch 159
Loss = 3.8526e-02, PNorm = 95.0911, GNorm = 0.0604, lr_0 = 7.7095e-04
Validation prc-auc = 0.415463
Epoch 160
Validation prc-auc = 0.432314
Epoch 161
Validation prc-auc = 0.436453
Epoch 162
Validation prc-auc = 0.440859
Epoch 163
Validation prc-auc = 0.428568
Epoch 164
Loss = 3.4340e-02, PNorm = 95.2158, GNorm = 0.9377, lr_0 = 7.6436e-04
Validation prc-auc = 0.435349
Epoch 165
Validation prc-auc = 0.422879
Epoch 166
Validation prc-auc = 0.420239
Epoch 167
Validation prc-auc = 0.405216
Epoch 168
Validation prc-auc = 0.416165
Epoch 169
Loss = 1.1325e-02, PNorm = 95.3269, GNorm = 0.9232, lr_0 = 7.5782e-04
Validation prc-auc = 0.418670
Epoch 170
Validation prc-auc = 0.426296
Epoch 171
Validation prc-auc = 0.426547
Epoch 172
Validation prc-auc = 0.426375
Epoch 173
Validation prc-auc = 0.425565
Epoch 174
Loss = 2.0936e-02, PNorm = 95.4529, GNorm = 0.4144, lr_0 = 7.5134e-04
Validation prc-auc = 0.423649
Epoch 175
Validation prc-auc = 0.420594
Epoch 176
Validation prc-auc = 0.400684
Epoch 177
Validation prc-auc = 0.420476
Epoch 178
Validation prc-auc = 0.411891
Epoch 179
Loss = 1.9563e-02, PNorm = 95.5936, GNorm = 0.6417, lr_0 = 7.4492e-04
Validation prc-auc = 0.412924
Epoch 180
Validation prc-auc = 0.420764
Epoch 181
Validation prc-auc = 0.423034
Epoch 182
Validation prc-auc = 0.429722
Epoch 183
Validation prc-auc = 0.432895
Epoch 184
Loss = 9.8591e-03, PNorm = 95.7103, GNorm = 0.4841, lr_0 = 7.3855e-04
Validation prc-auc = 0.436868
Epoch 185
Validation prc-auc = 0.439968
Epoch 186
Validation prc-auc = 0.440576
Epoch 187
Validation prc-auc = 0.441419
Epoch 188
Validation prc-auc = 0.443037
Epoch 189
Loss = 2.6623e-02, PNorm = 95.7929, GNorm = 1.0643, lr_0 = 7.3224e-04
Validation prc-auc = 0.441633
Epoch 190
Validation prc-auc = 0.441251
Epoch 191
Validation prc-auc = 0.440285
Epoch 192
Validation prc-auc = 0.445825
Epoch 193
Validation prc-auc = 0.440081
Epoch 194
Loss = 4.9089e-03, PNorm = 95.9054, GNorm = 0.0398, lr_0 = 7.2598e-04
Validation prc-auc = 0.432292
Epoch 195
Validation prc-auc = 0.430281
Epoch 196
Validation prc-auc = 0.429837
Epoch 197
Validation prc-auc = 0.440139
Epoch 198
Validation prc-auc = 0.439058
Epoch 199
Loss = 6.5086e-03, PNorm = 96.0016, GNorm = 0.2575, lr_0 = 7.1977e-04
Validation prc-auc = 0.438856
Epoch 200
Validation prc-auc = 0.440522
Epoch 201
Validation prc-auc = 0.441522
Epoch 202
Validation prc-auc = 0.421457
Epoch 203
Validation prc-auc = 0.423615
Epoch 204
Loss = 2.2446e-03, PNorm = 96.0880, GNorm = 0.0379, lr_0 = 7.1362e-04
Validation prc-auc = 0.433917
Epoch 205
Validation prc-auc = 0.435318
Epoch 206
Validation prc-auc = 0.435142
Epoch 207
Validation prc-auc = 0.433395
Epoch 208
Validation prc-auc = 0.433450
Epoch 209
Loss = 3.1818e-03, PNorm = 96.1513, GNorm = 0.1721, lr_0 = 7.0752e-04
Validation prc-auc = 0.425276
Epoch 210
Validation prc-auc = 0.431672
Epoch 211
Validation prc-auc = 0.429473
Epoch 212
Validation prc-auc = 0.412971
Epoch 213
Validation prc-auc = 0.411913
Epoch 214
Loss = 5.7183e-04, PNorm = 96.2123, GNorm = 0.0186, lr_0 = 7.0147e-04
Validation prc-auc = 0.413405
Epoch 215
Validation prc-auc = 0.410077
Epoch 216
Validation prc-auc = 0.392354
Epoch 217
Validation prc-auc = 0.388818
Epoch 218
Validation prc-auc = 0.386651
Epoch 219
Loss = 2.2740e-03, PNorm = 96.2795, GNorm = 0.1128, lr_0 = 6.9548e-04
Validation prc-auc = 0.405177
Epoch 220
Validation prc-auc = 0.403868
Epoch 221
Validation prc-auc = 0.397140
Epoch 222
Validation prc-auc = 0.396154
Epoch 223
Validation prc-auc = 0.399622
Epoch 224
Loss = 1.7042e-02, PNorm = 96.3273, GNorm = 0.9744, lr_0 = 6.8953e-04
Validation prc-auc = 0.401033
Epoch 225
Validation prc-auc = 0.397147
Epoch 226
Validation prc-auc = 0.387999
Epoch 227
Validation prc-auc = 0.395537
Epoch 228
Validation prc-auc = 0.395369
Epoch 229
Loss = 1.4669e-03, PNorm = 96.3865, GNorm = 0.1685, lr_0 = 6.8364e-04
Validation prc-auc = 0.410292
Epoch 230
Validation prc-auc = 0.410571
Epoch 231
Validation prc-auc = 0.411232
Epoch 232
Validation prc-auc = 0.411828
Epoch 233
Validation prc-auc = 0.414006
Epoch 234
Loss = 3.1635e-02, PNorm = 96.4327, GNorm = 0.0145, lr_0 = 6.7779e-04
Validation prc-auc = 0.414660
Epoch 235
Validation prc-auc = 0.423960
Epoch 236
Validation prc-auc = 0.413388
Epoch 237
Validation prc-auc = 0.409217
Epoch 238
Validation prc-auc = 0.412430
Epoch 239
Loss = 4.3702e-04, PNorm = 96.4793, GNorm = 0.0086, lr_0 = 6.7200e-04
Validation prc-auc = 0.412289
Epoch 240
Validation prc-auc = 0.411859
Epoch 241
Validation prc-auc = 0.415964
Epoch 242
Validation prc-auc = 0.413655
Epoch 243
Validation prc-auc = 0.418916
Epoch 244
Loss = 1.9146e-03, PNorm = 96.5186, GNorm = 0.1458, lr_0 = 6.6625e-04
Validation prc-auc = 0.428258
Epoch 245
Validation prc-auc = 0.428806
Epoch 246
Validation prc-auc = 0.416599
Epoch 247
Validation prc-auc = 0.416892
Epoch 248
Validation prc-auc = 0.416392
Epoch 249
Loss = 6.8575e-03, PNorm = 96.5763, GNorm = 0.0064, lr_0 = 6.6056e-04
Validation prc-auc = 0.401640
Epoch 250
Validation prc-auc = 0.401865
Epoch 251
Validation prc-auc = 0.409962
Epoch 252
Validation prc-auc = 0.415564
Epoch 253
Validation prc-auc = 0.417134
Epoch 254
Loss = 1.8980e-02, PNorm = 96.6361, GNorm = 0.4831, lr_0 = 6.5491e-04
Validation prc-auc = 0.417546
Epoch 255
Validation prc-auc = 0.423965
Epoch 256
Validation prc-auc = 0.415504
Epoch 257
Validation prc-auc = 0.401569
Epoch 258
Validation prc-auc = 0.387123
Epoch 259
Loss = 4.0862e-02, PNorm = 96.7272, GNorm = 0.0236, lr_0 = 6.4931e-04
Validation prc-auc = 0.398530
Epoch 260
Validation prc-auc = 0.383336
Epoch 261
Validation prc-auc = 0.399342
Epoch 262
Validation prc-auc = 0.393188
Epoch 263
Validation prc-auc = 0.401348
Epoch 264
Loss = 4.2296e-02, PNorm = 96.8572, GNorm = 1.7957, lr_0 = 6.4376e-04
Validation prc-auc = 0.404782
Epoch 265
Validation prc-auc = 0.399191
Epoch 266
Validation prc-auc = 0.402765
Epoch 267
Validation prc-auc = 0.406694
Epoch 268
Validation prc-auc = 0.405999
Epoch 269
Loss = 1.0300e-03, PNorm = 97.0880, GNorm = 0.0819, lr_0 = 6.3826e-04
Validation prc-auc = 0.408264
Epoch 270
Validation prc-auc = 0.393905
Epoch 271
Validation prc-auc = 0.378861
Epoch 272
Validation prc-auc = 0.385933
Epoch 273
Validation prc-auc = 0.390883
Epoch 274
Loss = 1.2510e-01, PNorm = 97.3087, GNorm = 0.1646, lr_0 = 6.3280e-04
Validation prc-auc = 0.384621
Epoch 275
Validation prc-auc = 0.384680
Epoch 276
Validation prc-auc = 0.369386
Epoch 277
Validation prc-auc = 0.377643
Epoch 278
Validation prc-auc = 0.287563
Epoch 279
Loss = 1.8751e-02, PNorm = 97.4947, GNorm = 1.5602, lr_0 = 6.2739e-04
Validation prc-auc = 0.290813
Epoch 280
Validation prc-auc = 0.365107
Epoch 281
Validation prc-auc = 0.395522
Epoch 282
Validation prc-auc = 0.410336
Epoch 283
Validation prc-auc = 0.408792
Epoch 284
Loss = 1.2057e-01, PNorm = 97.6132, GNorm = 3.3495, lr_0 = 6.2203e-04
Validation prc-auc = 0.414734
Epoch 285
Validation prc-auc = 0.445310
Epoch 286
Validation prc-auc = 0.434429
Epoch 287
Validation prc-auc = 0.438519
Epoch 288
Validation prc-auc = 0.441076
Epoch 289
Loss = 1.0998e-03, PNorm = 97.7123, GNorm = 0.0248, lr_0 = 6.1671e-04
Validation prc-auc = 0.448065
Epoch 290
Validation prc-auc = 0.449625
Epoch 291
Validation prc-auc = 0.450197
Epoch 292
Validation prc-auc = 0.431505
Epoch 293
Validation prc-auc = 0.422886
Epoch 294
Loss = 1.7240e-02, PNorm = 97.8111, GNorm = 0.9283, lr_0 = 6.1144e-04
Validation prc-auc = 0.425894
Epoch 295
Validation prc-auc = 0.406155
Epoch 296
Validation prc-auc = 0.413379
Epoch 297
Validation prc-auc = 0.413175
Epoch 298
Validation prc-auc = 0.409063
Epoch 299
Loss = 5.5281e-02, PNorm = 97.8832, GNorm = 2.0732, lr_0 = 6.0621e-04
Validation prc-auc = 0.402603
Model 0 best validation prc-auc = 0.466346 on epoch 47
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.371892
Ensemble test prc-auc = 0.371892
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.320942
Epoch 1
Validation prc-auc = 0.334801
Epoch 2
Validation prc-auc = 0.342303
Epoch 3
Validation prc-auc = 0.355049
Epoch 4
Loss = 5.2668e-01, PNorm = 87.5369, GNorm = 2.1584, lr_0 = 6.5000e-04
Validation prc-auc = 0.334318
Epoch 5
Validation prc-auc = 0.341213
Epoch 6
Validation prc-auc = 0.387171
Epoch 7
Validation prc-auc = 0.356974
Epoch 8
Validation prc-auc = 0.398002
Epoch 9
Loss = 4.0298e-01, PNorm = 87.9004, GNorm = 1.2218, lr_0 = 9.9743e-04
Validation prc-auc = 0.356924
Epoch 10
Validation prc-auc = 0.296474
Epoch 11
Validation prc-auc = 0.384636
Epoch 12
Validation prc-auc = 0.446734
Epoch 13
Validation prc-auc = 0.448231
Epoch 14
Loss = 3.3740e-01, PNorm = 88.2945, GNorm = 2.5695, lr_0 = 9.8890e-04
Validation prc-auc = 0.421330
Epoch 15
Validation prc-auc = 0.334450
Epoch 16
Validation prc-auc = 0.268383
Epoch 17
Validation prc-auc = 0.367854
Epoch 18
Validation prc-auc = 0.375399
Epoch 19
Loss = 3.0911e-01, PNorm = 88.6084, GNorm = 2.5989, lr_0 = 9.8045e-04
Validation prc-auc = 0.400320
Epoch 20
Validation prc-auc = 0.397036
Epoch 21
Validation prc-auc = 0.319581
Epoch 22
Validation prc-auc = 0.350295
Epoch 23
Validation prc-auc = 0.377755
Epoch 24
Loss = 3.8806e-01, PNorm = 88.8542, GNorm = 1.9643, lr_0 = 9.7207e-04
Validation prc-auc = 0.387444
Epoch 25
Validation prc-auc = 0.425506
Epoch 26
Validation prc-auc = 0.340688
Epoch 27
Validation prc-auc = 0.342019
Epoch 28
Validation prc-auc = 0.407304
Epoch 29
Loss = 2.8400e-01, PNorm = 89.0961, GNorm = 1.8109, lr_0 = 9.6376e-04
Validation prc-auc = 0.425419
Epoch 30
Validation prc-auc = 0.426805
Epoch 31
Validation prc-auc = 0.300293
Epoch 32
Validation prc-auc = 0.342920
Epoch 33
Validation prc-auc = 0.438620
Epoch 34
Loss = 3.8328e-01, PNorm = 89.3318, GNorm = 3.0605, lr_0 = 9.5552e-04
Validation prc-auc = 0.409620
Epoch 35
Validation prc-auc = 0.322599
Epoch 36
Validation prc-auc = 0.398576
Epoch 37
Validation prc-auc = 0.457164
Epoch 38
Validation prc-auc = 0.461868
Epoch 39
Loss = 3.2595e-01, PNorm = 89.5721, GNorm = 1.7344, lr_0 = 9.4735e-04
Validation prc-auc = 0.396709
Epoch 40
Validation prc-auc = 0.449447
Epoch 41
Validation prc-auc = 0.441710
Epoch 42
Validation prc-auc = 0.454587
Epoch 43
Validation prc-auc = 0.412147
Epoch 44
Loss = 3.3857e-01, PNorm = 89.7937, GNorm = 2.4925, lr_0 = 9.3925e-04
Validation prc-auc = 0.351717
Epoch 45
Validation prc-auc = 0.320802
Epoch 46
Validation prc-auc = 0.352714
Epoch 47
Validation prc-auc = 0.341825
Epoch 48
Validation prc-auc = 0.356324
Epoch 49
Loss = 2.5478e-01, PNorm = 90.0346, GNorm = 1.5570, lr_0 = 9.3122e-04
Validation prc-auc = 0.343985
Epoch 50
Validation prc-auc = 0.376940
Epoch 51
Validation prc-auc = 0.418450
Epoch 52
Validation prc-auc = 0.423662
Epoch 53
Validation prc-auc = 0.397363
Epoch 54
Loss = 2.5049e-01, PNorm = 90.2689, GNorm = 2.1010, lr_0 = 9.2326e-04
Validation prc-auc = 0.325568
Epoch 55
Validation prc-auc = 0.307217
Epoch 56
Validation prc-auc = 0.398154
Epoch 57
Validation prc-auc = 0.389899
Epoch 58
Validation prc-auc = 0.419775
Epoch 59
Loss = 1.4181e-01, PNorm = 90.5151, GNorm = 1.4760, lr_0 = 9.1537e-04
Validation prc-auc = 0.422165
Epoch 60
Validation prc-auc = 0.439340
Epoch 61
Validation prc-auc = 0.407261
Epoch 62
Validation prc-auc = 0.400210
Epoch 63
Validation prc-auc = 0.408044
Epoch 64
Loss = 1.4227e-01, PNorm = 90.7626, GNorm = 1.6007, lr_0 = 9.0754e-04
Validation prc-auc = 0.395475
Epoch 65
Validation prc-auc = 0.396295
Epoch 66
Validation prc-auc = 0.412140
Epoch 67
Validation prc-auc = 0.420530
Epoch 68
Validation prc-auc = 0.425869
Epoch 69
Loss = 1.2359e-01, PNorm = 91.0142, GNorm = 0.7659, lr_0 = 8.9978e-04
Validation prc-auc = 0.458839
Epoch 70
Validation prc-auc = 0.477348
Epoch 71
Validation prc-auc = 0.434547
Epoch 72
Validation prc-auc = 0.406923
Epoch 73
Validation prc-auc = 0.417289
Epoch 74
Loss = 1.4198e-01, PNorm = 91.2870, GNorm = 0.8389, lr_0 = 8.9209e-04
Validation prc-auc = 0.456671
Epoch 75
Validation prc-auc = 0.438849
Epoch 76
Validation prc-auc = 0.421406
Epoch 77
Validation prc-auc = 0.418155
Epoch 78
Validation prc-auc = 0.440563
Epoch 79
Loss = 1.1830e-01, PNorm = 91.5740, GNorm = 0.9565, lr_0 = 8.8447e-04
Validation prc-auc = 0.456699
Epoch 80
Validation prc-auc = 0.417070
Epoch 81
Validation prc-auc = 0.371895
Epoch 82
Validation prc-auc = 0.431081
Epoch 83
Validation prc-auc = 0.444650
Epoch 84
Loss = 2.6669e-02, PNorm = 91.8462, GNorm = 0.3389, lr_0 = 8.7691e-04
Validation prc-auc = 0.445529
Epoch 85
Validation prc-auc = 0.449448
Epoch 86
Validation prc-auc = 0.459185
Epoch 87
Validation prc-auc = 0.447546
Epoch 88
Validation prc-auc = 0.438248
Epoch 89
Loss = 1.0895e-01, PNorm = 92.0978, GNorm = 0.8411, lr_0 = 8.6941e-04
Validation prc-auc = 0.447352
Epoch 90
Validation prc-auc = 0.452667
Epoch 91
Validation prc-auc = 0.462858
Epoch 92
Validation prc-auc = 0.456647
Epoch 93
Validation prc-auc = 0.444222
Epoch 94
Loss = 4.1948e-02, PNorm = 92.3527, GNorm = 0.8106, lr_0 = 8.6198e-04
Validation prc-auc = 0.440218
Epoch 95
Validation prc-auc = 0.448901
Epoch 96
Validation prc-auc = 0.448149
Epoch 97
Validation prc-auc = 0.446291
Epoch 98
Validation prc-auc = 0.446375
Epoch 99
Loss = 4.2532e-02, PNorm = 92.6189, GNorm = 0.6072, lr_0 = 8.5461e-04
Validation prc-auc = 0.452350
Epoch 100
Validation prc-auc = 0.458851
Epoch 101
Validation prc-auc = 0.445937
Epoch 102
Validation prc-auc = 0.439483
Epoch 103
Validation prc-auc = 0.441012
Epoch 104
Loss = 8.6636e-02, PNorm = 92.8283, GNorm = 0.8361, lr_0 = 8.4730e-04
Validation prc-auc = 0.446064
Epoch 105
Validation prc-auc = 0.438314
Epoch 106
Validation prc-auc = 0.451972
Epoch 107
Validation prc-auc = 0.445667
Epoch 108
Validation prc-auc = 0.428379
Epoch 109
Loss = 8.2114e-02, PNorm = 93.0246, GNorm = 2.0635, lr_0 = 8.4006e-04
Validation prc-auc = 0.423540
Epoch 110
Validation prc-auc = 0.440028
Epoch 111
Validation prc-auc = 0.446254
Epoch 112
Validation prc-auc = 0.433721
Epoch 113
Validation prc-auc = 0.404959
Epoch 114
Loss = 6.4793e-02, PNorm = 93.2828, GNorm = 1.1199, lr_0 = 8.3288e-04
Validation prc-auc = 0.400418
Epoch 115
Validation prc-auc = 0.418830
Epoch 116
Validation prc-auc = 0.459596
Epoch 117
Validation prc-auc = 0.467024
Epoch 118
Validation prc-auc = 0.452908
Epoch 119
Loss = 1.2541e-01, PNorm = 93.5224, GNorm = 1.6787, lr_0 = 8.2576e-04
Validation prc-auc = 0.455458
Epoch 120
Validation prc-auc = 0.441895
Epoch 121
Validation prc-auc = 0.439720
Epoch 122
Validation prc-auc = 0.448410
Epoch 123
Validation prc-auc = 0.440994
Epoch 124
Loss = 7.6227e-02, PNorm = 93.7705, GNorm = 1.2931, lr_0 = 8.1870e-04
Validation prc-auc = 0.458020
Epoch 125
Validation prc-auc = 0.460790
Epoch 126
Validation prc-auc = 0.467565
Epoch 127
Validation prc-auc = 0.464914
Epoch 128
Validation prc-auc = 0.461171
Epoch 129
Loss = 5.5371e-02, PNorm = 94.0101, GNorm = 0.6387, lr_0 = 8.1170e-04
Validation prc-auc = 0.454613
Epoch 130
Validation prc-auc = 0.457385
Epoch 131
Validation prc-auc = 0.459594
Epoch 132
Validation prc-auc = 0.460792
Epoch 133
Validation prc-auc = 0.470386
Epoch 134
Loss = 6.3177e-02, PNorm = 94.1914, GNorm = 0.9421, lr_0 = 8.0476e-04
Validation prc-auc = 0.468311
Epoch 135
Validation prc-auc = 0.458219
Epoch 136
Validation prc-auc = 0.459470
Epoch 137
Validation prc-auc = 0.444437
Epoch 138
Validation prc-auc = 0.443441
Epoch 139
Loss = 4.9829e-03, PNorm = 94.3857, GNorm = 0.1820, lr_0 = 7.9788e-04
Validation prc-auc = 0.440368
Epoch 140
Validation prc-auc = 0.438795
Epoch 141
Validation prc-auc = 0.430945
Epoch 142
Validation prc-auc = 0.443048
Epoch 143
Validation prc-auc = 0.421277
Epoch 144
Loss = 7.3522e-02, PNorm = 94.5813, GNorm = 1.4314, lr_0 = 7.9106e-04
Validation prc-auc = 0.446578
Epoch 145
Validation prc-auc = 0.450898
Epoch 146
Validation prc-auc = 0.453523
Epoch 147
Validation prc-auc = 0.429577
Epoch 148
Validation prc-auc = 0.439975
Epoch 149
Loss = 4.7050e-02, PNorm = 94.7686, GNorm = 3.0667, lr_0 = 7.8430e-04
Validation prc-auc = 0.436165
Epoch 150
Validation prc-auc = 0.455563
Epoch 151
Validation prc-auc = 0.461642
Epoch 152
Validation prc-auc = 0.469109
Epoch 153
Validation prc-auc = 0.447842
Epoch 154
Loss = 7.2868e-02, PNorm = 94.9992, GNorm = 3.1319, lr_0 = 7.7759e-04
Validation prc-auc = 0.454218
Epoch 155
Validation prc-auc = 0.448286
Epoch 156
Validation prc-auc = 0.438601
Epoch 157
Validation prc-auc = 0.412753
Epoch 158
Validation prc-auc = 0.426733
Epoch 159
Loss = 3.8615e-02, PNorm = 95.3120, GNorm = 0.9913, lr_0 = 7.7095e-04
Validation prc-auc = 0.421147
Epoch 160
Validation prc-auc = 0.414417
Epoch 161
Validation prc-auc = 0.424594
Epoch 162
Validation prc-auc = 0.415929
Epoch 163
Validation prc-auc = 0.411921
Epoch 164
Loss = 7.7259e-02, PNorm = 95.5828, GNorm = 1.2924, lr_0 = 7.6436e-04
Validation prc-auc = 0.359058
Epoch 165
Validation prc-auc = 0.379146
Epoch 166
Validation prc-auc = 0.384014
Epoch 167
Validation prc-auc = 0.420094
Epoch 168
Validation prc-auc = 0.427964
Epoch 169
Loss = 5.5369e-02, PNorm = 95.8820, GNorm = 0.8454, lr_0 = 7.5782e-04
Validation prc-auc = 0.448873
Epoch 170
Validation prc-auc = 0.451092
Epoch 171
Validation prc-auc = 0.438892
Epoch 172
Validation prc-auc = 0.388407
Epoch 173
Validation prc-auc = 0.387278
Epoch 174
Loss = 2.8626e-02, PNorm = 96.1554, GNorm = 0.9571, lr_0 = 7.5134e-04
Validation prc-auc = 0.376374
Epoch 175
Validation prc-auc = 0.384094
Epoch 176
Validation prc-auc = 0.421571
Epoch 177
Validation prc-auc = 0.438222
Epoch 178
Validation prc-auc = 0.445378
Epoch 179
Loss = 7.6221e-03, PNorm = 96.3573, GNorm = 0.1996, lr_0 = 7.4492e-04
Validation prc-auc = 0.444971
Epoch 180
Validation prc-auc = 0.445131
Epoch 181
Validation prc-auc = 0.444840
Epoch 182
Validation prc-auc = 0.447867
Epoch 183
Validation prc-auc = 0.450823
Epoch 184
Loss = 7.1207e-02, PNorm = 96.5120, GNorm = 1.6571, lr_0 = 7.3855e-04
Validation prc-auc = 0.470305
Epoch 185
Validation prc-auc = 0.446296
Epoch 186
Validation prc-auc = 0.435948
Epoch 187
Validation prc-auc = 0.416488
Epoch 188
Validation prc-auc = 0.415362
Epoch 189
Loss = 3.9637e-02, PNorm = 96.6964, GNorm = 1.1877, lr_0 = 7.3224e-04
Validation prc-auc = 0.407510
Epoch 190
Validation prc-auc = 0.406029
Epoch 191
Validation prc-auc = 0.369241
Epoch 192
Validation prc-auc = 0.360494
Epoch 193
Validation prc-auc = 0.350831
Epoch 194
Loss = 9.7718e-02, PNorm = 96.8796, GNorm = 1.3100, lr_0 = 7.2598e-04
Validation prc-auc = 0.375965
Epoch 195
Validation prc-auc = 0.460596
Epoch 196
Validation prc-auc = 0.469077
Epoch 197
Validation prc-auc = 0.455733
Epoch 198
Validation prc-auc = 0.455067
Epoch 199
Loss = 1.9551e-02, PNorm = 97.0770, GNorm = 0.0991, lr_0 = 7.1977e-04
Validation prc-auc = 0.454216
Epoch 200
Validation prc-auc = 0.438746
Epoch 201
Validation prc-auc = 0.440272
Epoch 202
Validation prc-auc = 0.439838
Epoch 203
Validation prc-auc = 0.423975
Epoch 204
Loss = 2.7262e-02, PNorm = 97.2651, GNorm = 0.9819, lr_0 = 7.1362e-04
Validation prc-auc = 0.446588
Epoch 205
Validation prc-auc = 0.456495
Epoch 206
Validation prc-auc = 0.454624
Epoch 207
Validation prc-auc = 0.451204
Epoch 208
Validation prc-auc = 0.461857
Epoch 209
Loss = 2.2378e-02, PNorm = 97.4091, GNorm = 0.8377, lr_0 = 7.0752e-04
Validation prc-auc = 0.460931
Epoch 210
Validation prc-auc = 0.461779
Epoch 211
Validation prc-auc = 0.462210
Epoch 212
Validation prc-auc = 0.463524
Epoch 213
Validation prc-auc = 0.462103
Epoch 214
Loss = 2.6111e-02, PNorm = 97.5050, GNorm = 0.7168, lr_0 = 7.0147e-04
Validation prc-auc = 0.459882
Epoch 215
Validation prc-auc = 0.457563
Epoch 216
Validation prc-auc = 0.456161
Epoch 217
Validation prc-auc = 0.447154
Epoch 218
Validation prc-auc = 0.432112
Epoch 219
Loss = 2.1303e-02, PNorm = 97.5847, GNorm = 0.4333, lr_0 = 6.9548e-04
Validation prc-auc = 0.432035
Epoch 220
Validation prc-auc = 0.439325
Epoch 221
Validation prc-auc = 0.455574
Epoch 222
Validation prc-auc = 0.441572
Epoch 223
Validation prc-auc = 0.442252
Epoch 224
Loss = 2.0390e-03, PNorm = 97.6369, GNorm = 0.0164, lr_0 = 6.8953e-04
Validation prc-auc = 0.443645
Epoch 225
Validation prc-auc = 0.443350
Epoch 226
Validation prc-auc = 0.460604
Epoch 227
Validation prc-auc = 0.463028
Epoch 228
Validation prc-auc = 0.463653
Epoch 229
Loss = 2.9036e-02, PNorm = 97.6748, GNorm = 0.0715, lr_0 = 6.8364e-04
Validation prc-auc = 0.439688
Epoch 230
Validation prc-auc = 0.434539
Epoch 231
Validation prc-auc = 0.448215
Epoch 232
Validation prc-auc = 0.446100
Epoch 233
Validation prc-auc = 0.446767
Epoch 234
Loss = 1.3403e-02, PNorm = 97.7740, GNorm = 0.5348, lr_0 = 6.7779e-04
Validation prc-auc = 0.447906
Epoch 235
Validation prc-auc = 0.450006
Epoch 236
Validation prc-auc = 0.452183
Epoch 237
Validation prc-auc = 0.453225
Epoch 238
Validation prc-auc = 0.441621
Epoch 239
Loss = 1.2328e-02, PNorm = 97.8842, GNorm = 0.7442, lr_0 = 6.7200e-04
Validation prc-auc = 0.426780
Epoch 240
Validation prc-auc = 0.430624
Epoch 241
Validation prc-auc = 0.433482
Epoch 242
Validation prc-auc = 0.452487
Epoch 243
Validation prc-auc = 0.452133
Epoch 244
Loss = 2.8147e-03, PNorm = 98.0058, GNorm = 0.0020, lr_0 = 6.6625e-04
Validation prc-auc = 0.451768
Epoch 245
Validation prc-auc = 0.465639
Epoch 246
Validation prc-auc = 0.461941
Epoch 247
Validation prc-auc = 0.459267
Epoch 248
Validation prc-auc = 0.450709
Epoch 249
Loss = 6.2688e-03, PNorm = 98.1278, GNorm = 0.3730, lr_0 = 6.6056e-04
Validation prc-auc = 0.443122
Epoch 250
Validation prc-auc = 0.433191
Epoch 251
Validation prc-auc = 0.444823
Epoch 252
Validation prc-auc = 0.454494
Epoch 253
Validation prc-auc = 0.453372
Epoch 254
Loss = 1.0328e-03, PNorm = 98.2751, GNorm = 0.0109, lr_0 = 6.5491e-04
Validation prc-auc = 0.438549
Epoch 255
Validation prc-auc = 0.437994
Epoch 256
Validation prc-auc = 0.438041
Epoch 257
Validation prc-auc = 0.429459
Epoch 258
Validation prc-auc = 0.430046
Epoch 259
Loss = 9.8773e-02, PNorm = 98.3897, GNorm = 0.5652, lr_0 = 6.4931e-04
Validation prc-auc = 0.431158
Epoch 260
Validation prc-auc = 0.433701
Epoch 261
Validation prc-auc = 0.444294
Epoch 262
Validation prc-auc = 0.448582
Epoch 263
Validation prc-auc = 0.452613
Epoch 264
Loss = 3.6069e-03, PNorm = 98.4838, GNorm = 0.0278, lr_0 = 6.4376e-04
Validation prc-auc = 0.452254
Epoch 265
Validation prc-auc = 0.454838
Epoch 266
Validation prc-auc = 0.455529
Epoch 267
Validation prc-auc = 0.454898
Epoch 268
Validation prc-auc = 0.452018
Epoch 269
Loss = 9.8581e-04, PNorm = 98.5439, GNorm = 0.0300, lr_0 = 6.3826e-04
Validation prc-auc = 0.451068
Epoch 270
Validation prc-auc = 0.448471
Epoch 271
Validation prc-auc = 0.448223
Epoch 272
Validation prc-auc = 0.450510
Epoch 273
Validation prc-auc = 0.453236
Epoch 274
Loss = 1.0143e-02, PNorm = 98.5949, GNorm = 0.6090, lr_0 = 6.3280e-04
Validation prc-auc = 0.432894
Epoch 275
Validation prc-auc = 0.430198
Epoch 276
Validation prc-auc = 0.429029
Epoch 277
Validation prc-auc = 0.427534
Epoch 278
Validation prc-auc = 0.426462
Epoch 279
Loss = 3.7233e-03, PNorm = 98.6418, GNorm = 0.3036, lr_0 = 6.2739e-04
Validation prc-auc = 0.425101
Epoch 280
Validation prc-auc = 0.424184
Epoch 281
Validation prc-auc = 0.420118
Epoch 282
Validation prc-auc = 0.425445
Epoch 283
Validation prc-auc = 0.427064
Epoch 284
Loss = 1.3519e-02, PNorm = 98.7176, GNorm = 0.8710, lr_0 = 6.2203e-04
Validation prc-auc = 0.432368
Epoch 285
Validation prc-auc = 0.444043
Epoch 286
Validation prc-auc = 0.442124
Epoch 287
Validation prc-auc = 0.434702
Epoch 288
Validation prc-auc = 0.441525
Epoch 289
Loss = 3.6193e-03, PNorm = 98.8450, GNorm = 0.0343, lr_0 = 6.1671e-04
Validation prc-auc = 0.440747
Epoch 290
Validation prc-auc = 0.437600
Epoch 291
Validation prc-auc = 0.437136
Epoch 292
Validation prc-auc = 0.436762
Epoch 293
Validation prc-auc = 0.438154
Epoch 294
Loss = 2.7704e-03, PNorm = 98.9384, GNorm = 0.0336, lr_0 = 6.1144e-04
Validation prc-auc = 0.439484
Epoch 295
Validation prc-auc = 0.442546
Epoch 296
Validation prc-auc = 0.443071
Epoch 297
Validation prc-auc = 0.442538
Epoch 298
Validation prc-auc = 0.443235
Epoch 299
Loss = 1.8256e-03, PNorm = 98.9929, GNorm = 0.0411, lr_0 = 6.0621e-04
Validation prc-auc = 0.443785
Model 0 best validation prc-auc = 0.477348 on epoch 70
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.234045
Ensemble test prc-auc = 0.234045
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.406327
Epoch 1
Validation prc-auc = 0.437636
Epoch 2
Validation prc-auc = 0.381028
Epoch 3
Validation prc-auc = 0.180187
Epoch 4
Loss = 6.6223e-01, PNorm = 87.5393, GNorm = 2.0274, lr_0 = 6.5000e-04
Validation prc-auc = 0.357518
Epoch 5
Validation prc-auc = 0.433536
Epoch 6
Validation prc-auc = 0.426368
Epoch 7
Validation prc-auc = 0.404270
Epoch 8
Validation prc-auc = 0.346073
Epoch 9
Loss = 5.0517e-01, PNorm = 87.8985, GNorm = 1.9827, lr_0 = 9.9743e-04
Validation prc-auc = 0.283579
Epoch 10
Validation prc-auc = 0.397992
Epoch 11
Validation prc-auc = 0.422256
Epoch 12
Validation prc-auc = 0.388518
Epoch 13
Validation prc-auc = 0.305180
Epoch 14
Loss = 3.5156e-01, PNorm = 88.2847, GNorm = 2.5368, lr_0 = 9.8890e-04
Validation prc-auc = 0.336479
Epoch 15
Validation prc-auc = 0.410921
Epoch 16
Validation prc-auc = 0.428988
Epoch 17
Validation prc-auc = 0.339364
Epoch 18
Validation prc-auc = 0.299394
Epoch 19
Loss = 2.3854e-01, PNorm = 88.5703, GNorm = 1.0679, lr_0 = 9.8045e-04
Validation prc-auc = 0.342923
Epoch 20
Validation prc-auc = 0.390644
Epoch 21
Validation prc-auc = 0.396229
Epoch 22
Validation prc-auc = 0.323285
Epoch 23
Validation prc-auc = 0.326330
Epoch 24
Loss = 4.0337e-01, PNorm = 88.8058, GNorm = 1.4244, lr_0 = 9.7207e-04
Validation prc-auc = 0.395556
Epoch 25
Validation prc-auc = 0.358353
Epoch 26
Validation prc-auc = 0.183542
Epoch 27
Validation prc-auc = 0.236165
Epoch 28
Validation prc-auc = 0.406727
Epoch 29
Loss = 3.8103e-01, PNorm = 89.0472, GNorm = 1.5811, lr_0 = 9.6376e-04
Validation prc-auc = 0.443118
Epoch 30
Validation prc-auc = 0.446016
Epoch 31
Validation prc-auc = 0.429564
Epoch 32
Validation prc-auc = 0.413696
Epoch 33
Validation prc-auc = 0.383522
Epoch 34
Loss = 2.6715e-01, PNorm = 89.2961, GNorm = 1.0898, lr_0 = 9.5552e-04
Validation prc-auc = 0.349119
Epoch 35
Validation prc-auc = 0.355297
Epoch 36
Validation prc-auc = 0.364760
Epoch 37
Validation prc-auc = 0.394280
Epoch 38
Validation prc-auc = 0.344284
Epoch 39
Loss = 1.8815e-01, PNorm = 89.5533, GNorm = 1.2845, lr_0 = 9.4735e-04
Validation prc-auc = 0.262877
Epoch 40
Validation prc-auc = 0.271608
Epoch 41
Validation prc-auc = 0.413829
Epoch 42
Validation prc-auc = 0.405337
Epoch 43
Validation prc-auc = 0.346218
Epoch 44
Loss = 2.7042e-01, PNorm = 89.7696, GNorm = 2.1846, lr_0 = 9.3925e-04
Validation prc-auc = 0.391204
Epoch 45
Validation prc-auc = 0.405479
Epoch 46
Validation prc-auc = 0.416916
Epoch 47
Validation prc-auc = 0.420569
Epoch 48
Validation prc-auc = 0.434131
Epoch 49
Loss = 1.4763e-01, PNorm = 90.0230, GNorm = 0.8199, lr_0 = 9.3122e-04
Validation prc-auc = 0.439329
Epoch 50
Validation prc-auc = 0.443153
Epoch 51
Validation prc-auc = 0.448223
Epoch 52
Validation prc-auc = 0.440898
Epoch 53
Validation prc-auc = 0.409753
Epoch 54
Loss = 1.3390e-01, PNorm = 90.2812, GNorm = 1.1503, lr_0 = 9.2326e-04
Validation prc-auc = 0.425645
Epoch 55
Validation prc-auc = 0.421329
Epoch 56
Validation prc-auc = 0.437937
Epoch 57
Validation prc-auc = 0.427557
Epoch 58
Validation prc-auc = 0.410265
Epoch 59
Loss = 1.7241e-01, PNorm = 90.5564, GNorm = 1.6785, lr_0 = 9.1537e-04
Validation prc-auc = 0.408640
Epoch 60
Validation prc-auc = 0.401181
Epoch 61
Validation prc-auc = 0.404801
Epoch 62
Validation prc-auc = 0.385098
Epoch 63
Validation prc-auc = 0.432710
Epoch 64
Loss = 1.6292e-01, PNorm = 90.8306, GNorm = 1.2519, lr_0 = 9.0754e-04
Validation prc-auc = 0.468971
Epoch 65
Validation prc-auc = 0.475159
Epoch 66
Validation prc-auc = 0.478304
Epoch 67
Validation prc-auc = 0.453666
Epoch 68
Validation prc-auc = 0.452158
Epoch 69
Loss = 8.0765e-02, PNorm = 91.1396, GNorm = 0.4647, lr_0 = 8.9978e-04
Validation prc-auc = 0.445276
Epoch 70
Validation prc-auc = 0.445963
Epoch 71
Validation prc-auc = 0.440896
Epoch 72
Validation prc-auc = 0.435874
Epoch 73
Validation prc-auc = 0.441431
Epoch 74
Loss = 6.5064e-02, PNorm = 91.4099, GNorm = 0.7637, lr_0 = 8.9209e-04
Validation prc-auc = 0.434577
Epoch 75
Validation prc-auc = 0.424236
Epoch 76
Validation prc-auc = 0.440675
Epoch 77
Validation prc-auc = 0.434706
Epoch 78
Validation prc-auc = 0.425121
Epoch 79
Loss = 1.1926e-01, PNorm = 91.6345, GNorm = 1.3871, lr_0 = 8.8447e-04
Validation prc-auc = 0.427460
Epoch 80
Validation prc-auc = 0.427575
Epoch 81
Validation prc-auc = 0.424668
Epoch 82
Validation prc-auc = 0.420938
Epoch 83
Validation prc-auc = 0.417861
Epoch 84
Loss = 1.5456e-01, PNorm = 91.8570, GNorm = 1.4819, lr_0 = 8.7691e-04
Validation prc-auc = 0.419881
Epoch 85
Validation prc-auc = 0.436680
Epoch 86
Validation prc-auc = 0.430086
Epoch 87
Validation prc-auc = 0.408478
Epoch 88
Validation prc-auc = 0.422121
Epoch 89
Loss = 9.0838e-02, PNorm = 92.1282, GNorm = 0.6912, lr_0 = 8.6941e-04
Validation prc-auc = 0.442672
Epoch 90
Validation prc-auc = 0.429970
Epoch 91
Validation prc-auc = 0.420175
Epoch 92
Validation prc-auc = 0.406199
Epoch 93
Validation prc-auc = 0.440682
Epoch 94
Loss = 1.1147e-01, PNorm = 92.4247, GNorm = 1.7485, lr_0 = 8.6198e-04
Validation prc-auc = 0.458850
Epoch 95
Validation prc-auc = 0.428892
Epoch 96
Validation prc-auc = 0.432905
Epoch 97
Validation prc-auc = 0.441453
Epoch 98
Validation prc-auc = 0.451693
Epoch 99
Loss = 7.1129e-02, PNorm = 92.7137, GNorm = 1.3455, lr_0 = 8.5461e-04
Validation prc-auc = 0.418773
Epoch 100
Validation prc-auc = 0.423765
Epoch 101
Validation prc-auc = 0.417672
Epoch 102
Validation prc-auc = 0.416756
Epoch 103
Validation prc-auc = 0.424468
Epoch 104
Loss = 4.4185e-02, PNorm = 92.9832, GNorm = 1.5268, lr_0 = 8.4730e-04
Validation prc-auc = 0.435279
Epoch 105
Validation prc-auc = 0.433514
Epoch 106
Validation prc-auc = 0.427020
Epoch 107
Validation prc-auc = 0.431165
Epoch 108
Validation prc-auc = 0.422600
Epoch 109
Loss = 9.9093e-02, PNorm = 93.2345, GNorm = 1.6775, lr_0 = 8.4006e-04
Validation prc-auc = 0.426915
Epoch 110
Validation prc-auc = 0.416601
Epoch 111
Validation prc-auc = 0.422133
Epoch 112
Validation prc-auc = 0.407035
Epoch 113
Validation prc-auc = 0.419468
Epoch 114
Loss = 1.1007e-01, PNorm = 93.4360, GNorm = 1.6447, lr_0 = 8.3288e-04
Validation prc-auc = 0.425428
Epoch 115
Validation prc-auc = 0.429570
Epoch 116
Validation prc-auc = 0.447366
Epoch 117
Validation prc-auc = 0.440729
Epoch 118
Validation prc-auc = 0.409605
Epoch 119
Loss = 4.6127e-02, PNorm = 93.7003, GNorm = 0.5956, lr_0 = 8.2576e-04
Validation prc-auc = 0.395315
Epoch 120
Validation prc-auc = 0.398176
Epoch 121
Validation prc-auc = 0.415238
Epoch 122
Validation prc-auc = 0.381969
Epoch 123
Validation prc-auc = 0.408850
Epoch 124
Loss = 4.7335e-02, PNorm = 93.9628, GNorm = 0.4836, lr_0 = 8.1870e-04
Validation prc-auc = 0.414409
Epoch 125
Validation prc-auc = 0.434625
Epoch 126
Validation prc-auc = 0.437003
Epoch 127
Validation prc-auc = 0.453353
Epoch 128
Validation prc-auc = 0.456842
Epoch 129
Loss = 6.0522e-02, PNorm = 94.1814, GNorm = 0.3187, lr_0 = 8.1170e-04
Validation prc-auc = 0.468323
Epoch 130
Validation prc-auc = 0.464649
Epoch 131
Validation prc-auc = 0.449437
Epoch 132
Validation prc-auc = 0.459406
Epoch 133
Validation prc-auc = 0.450597
Epoch 134
Loss = 5.6280e-03, PNorm = 94.3820, GNorm = 0.1257, lr_0 = 8.0476e-04
Validation prc-auc = 0.449177
Epoch 135
Validation prc-auc = 0.433762
Epoch 136
Validation prc-auc = 0.413306
Epoch 137
Validation prc-auc = 0.417778
Epoch 138
Validation prc-auc = 0.413215
Epoch 139
Loss = 6.3636e-02, PNorm = 94.5300, GNorm = 0.9795, lr_0 = 7.9788e-04
Validation prc-auc = 0.423715
Epoch 140
Validation prc-auc = 0.436919
Epoch 141
Validation prc-auc = 0.452068
Epoch 142
Validation prc-auc = 0.439384
Epoch 143
Validation prc-auc = 0.445309
Epoch 144
Loss = 6.4431e-02, PNorm = 94.6925, GNorm = 1.0578, lr_0 = 7.9106e-04
Validation prc-auc = 0.428636
Epoch 145
Validation prc-auc = 0.424634
Epoch 146
Validation prc-auc = 0.422918
Epoch 147
Validation prc-auc = 0.423660
Epoch 148
Validation prc-auc = 0.433937
Epoch 149
Loss = 6.1418e-02, PNorm = 94.8678, GNorm = 0.2708, lr_0 = 7.8430e-04
Validation prc-auc = 0.437679
Epoch 150
Validation prc-auc = 0.453994
Epoch 151
Validation prc-auc = 0.435669
Epoch 152
Validation prc-auc = 0.441200
Epoch 153
Validation prc-auc = 0.435032
Epoch 154
Loss = 8.3916e-02, PNorm = 95.0325, GNorm = 1.1017, lr_0 = 7.7759e-04
Validation prc-auc = 0.413941
Epoch 155
Validation prc-auc = 0.375463
Epoch 156
Validation prc-auc = 0.379215
Epoch 157
Validation prc-auc = 0.413597
Epoch 158
Validation prc-auc = 0.406346
Epoch 159
Loss = 1.7399e-02, PNorm = 95.2206, GNorm = 0.6177, lr_0 = 7.7095e-04
Validation prc-auc = 0.420586
Epoch 160
Validation prc-auc = 0.436623
Epoch 161
Validation prc-auc = 0.435348
Epoch 162
Validation prc-auc = 0.428205
Epoch 163
Validation prc-auc = 0.428563
Epoch 164
Loss = 8.9079e-03, PNorm = 95.3843, GNorm = 0.4367, lr_0 = 7.6436e-04
Validation prc-auc = 0.427788
Epoch 165
Validation prc-auc = 0.429606
Epoch 166
Validation prc-auc = 0.427702
Epoch 167
Validation prc-auc = 0.425830
Epoch 168
Validation prc-auc = 0.426106
Epoch 169
Loss = 1.4182e-02, PNorm = 95.4878, GNorm = 0.2669, lr_0 = 7.5782e-04
Validation prc-auc = 0.425447
Epoch 170
Validation prc-auc = 0.426252
Epoch 171
Validation prc-auc = 0.412232
Epoch 172
Validation prc-auc = 0.416346
Epoch 173
Validation prc-auc = 0.421826
Epoch 174
Loss = 1.3560e-02, PNorm = 95.5690, GNorm = 0.6274, lr_0 = 7.5134e-04
Validation prc-auc = 0.426345
Epoch 175
Validation prc-auc = 0.425624
Epoch 176
Validation prc-auc = 0.411711
Epoch 177
Validation prc-auc = 0.415922
Epoch 178
Validation prc-auc = 0.432749
Epoch 179
Loss = 3.4886e-02, PNorm = 95.6542, GNorm = 2.2575, lr_0 = 7.4492e-04
Validation prc-auc = 0.436600
Epoch 180
Validation prc-auc = 0.430489
Epoch 181
Validation prc-auc = 0.426430
Epoch 182
Validation prc-auc = 0.431439
Epoch 183
Validation prc-auc = 0.438804
Epoch 184
Loss = 1.3805e-02, PNorm = 95.8025, GNorm = 0.3905, lr_0 = 7.3855e-04
Validation prc-auc = 0.442327
Epoch 185
Validation prc-auc = 0.458108
Epoch 186
Validation prc-auc = 0.438206
Epoch 187
Validation prc-auc = 0.435563
Epoch 188
Validation prc-auc = 0.426024
Epoch 189
Loss = 1.0759e-01, PNorm = 95.9834, GNorm = 1.6575, lr_0 = 7.3224e-04
Validation prc-auc = 0.431844
Epoch 190
Validation prc-auc = 0.445692
Epoch 191
Validation prc-auc = 0.440180
Epoch 192
Validation prc-auc = 0.426884
Epoch 193
Validation prc-auc = 0.434843
Epoch 194
Loss = 5.4674e-02, PNorm = 96.1931, GNorm = 1.0474, lr_0 = 7.2598e-04
Validation prc-auc = 0.436967
Epoch 195
Validation prc-auc = 0.440307
Epoch 196
Validation prc-auc = 0.447675
Epoch 197
Validation prc-auc = 0.448739
Epoch 198
Validation prc-auc = 0.442713
Epoch 199
Loss = 3.4460e-02, PNorm = 96.4260, GNorm = 0.6569, lr_0 = 7.1977e-04
Validation prc-auc = 0.430676
Epoch 200
Validation prc-auc = 0.399009
Epoch 201
Validation prc-auc = 0.410074
Epoch 202
Validation prc-auc = 0.436818
Epoch 203
Validation prc-auc = 0.429812
Epoch 204
Loss = 1.1761e-02, PNorm = 96.7126, GNorm = 0.4865, lr_0 = 7.1362e-04
Validation prc-auc = 0.434964
Epoch 205
Validation prc-auc = 0.436953
Epoch 206
Validation prc-auc = 0.444877
Epoch 207
Validation prc-auc = 0.449347
Epoch 208
Validation prc-auc = 0.448833
Epoch 209
Loss = 7.0327e-03, PNorm = 96.9642, GNorm = 0.3474, lr_0 = 7.0752e-04
Validation prc-auc = 0.448316
Epoch 210
Validation prc-auc = 0.445644
Epoch 211
Validation prc-auc = 0.438618
Epoch 212
Validation prc-auc = 0.439208
Epoch 213
Validation prc-auc = 0.432549
Epoch 214
Loss = 4.4925e-02, PNorm = 97.1459, GNorm = 1.8840, lr_0 = 7.0147e-04
Validation prc-auc = 0.440063
Epoch 215
Validation prc-auc = 0.452072
Epoch 216
Validation prc-auc = 0.452565
Epoch 217
Validation prc-auc = 0.451796
Epoch 218
Validation prc-auc = 0.447457
Epoch 219
Loss = 6.6524e-03, PNorm = 97.3071, GNorm = 0.5095, lr_0 = 6.9548e-04
Validation prc-auc = 0.442555
Epoch 220
Validation prc-auc = 0.431019
Epoch 221
Validation prc-auc = 0.399091
Epoch 222
Validation prc-auc = 0.395067
Epoch 223
Validation prc-auc = 0.394913
Epoch 224
Loss = 4.3579e-02, PNorm = 97.4527, GNorm = 0.2642, lr_0 = 6.8953e-04
Validation prc-auc = 0.388392
Epoch 225
Validation prc-auc = 0.390471
Epoch 226
Validation prc-auc = 0.407551
Epoch 227
Validation prc-auc = 0.422154
Epoch 228
Validation prc-auc = 0.416325
Epoch 229
Loss = 7.8572e-03, PNorm = 97.5912, GNorm = 0.3626, lr_0 = 6.8364e-04
Validation prc-auc = 0.414137
Epoch 230
Validation prc-auc = 0.409556
Epoch 231
Validation prc-auc = 0.401857
Epoch 232
Validation prc-auc = 0.399396
Epoch 233
Validation prc-auc = 0.402855
Epoch 234
Loss = 1.5183e-03, PNorm = 97.7431, GNorm = 0.0302, lr_0 = 6.7779e-04
Validation prc-auc = 0.409734
Epoch 235
Validation prc-auc = 0.414563
Epoch 236
Validation prc-auc = 0.433987
Epoch 237
Validation prc-auc = 0.442664
Epoch 238
Validation prc-auc = 0.444179
Epoch 239
Loss = 6.8414e-03, PNorm = 97.8370, GNorm = 0.3118, lr_0 = 6.7200e-04
Validation prc-auc = 0.445180
Epoch 240
Validation prc-auc = 0.445957
Epoch 241
Validation prc-auc = 0.443750
Epoch 242
Validation prc-auc = 0.441187
Epoch 243
Validation prc-auc = 0.415511
Epoch 244
Loss = 6.1260e-02, PNorm = 97.9289, GNorm = 0.2264, lr_0 = 6.6625e-04
Validation prc-auc = 0.400009
Epoch 245
Validation prc-auc = 0.399405
Epoch 246
Validation prc-auc = 0.405779
Epoch 247
Validation prc-auc = 0.421244
Epoch 248
Validation prc-auc = 0.419660
Epoch 249
Loss = 5.3215e-02, PNorm = 98.0402, GNorm = 1.3461, lr_0 = 6.6056e-04
Validation prc-auc = 0.420871
Epoch 250
Validation prc-auc = 0.422878
Epoch 251
Validation prc-auc = 0.424796
Epoch 252
Validation prc-auc = 0.424777
Epoch 253
Validation prc-auc = 0.426088
Epoch 254
Loss = 1.2812e-02, PNorm = 98.1281, GNorm = 0.3720, lr_0 = 6.5491e-04
Validation prc-auc = 0.428467
Epoch 255
Validation prc-auc = 0.427923
Epoch 256
Validation prc-auc = 0.427135
Epoch 257
Validation prc-auc = 0.426756
Epoch 258
Validation prc-auc = 0.426349
Epoch 259
Loss = 1.3987e-02, PNorm = 98.1785, GNorm = 0.0521, lr_0 = 6.4931e-04
Validation prc-auc = 0.426132
Epoch 260
Validation prc-auc = 0.428056
Epoch 261
Validation prc-auc = 0.425984
Epoch 262
Validation prc-auc = 0.424895
Epoch 263
Validation prc-auc = 0.419936
Epoch 264
Loss = 1.6048e-03, PNorm = 98.2181, GNorm = 0.0331, lr_0 = 6.4376e-04
Validation prc-auc = 0.419601
Epoch 265
Validation prc-auc = 0.423838
Epoch 266
Validation prc-auc = 0.423714
Epoch 267
Validation prc-auc = 0.423846
Epoch 268
Validation prc-auc = 0.428959
Epoch 269
Loss = 2.3936e-03, PNorm = 98.2490, GNorm = 0.0935, lr_0 = 6.3826e-04
Validation prc-auc = 0.428513
Epoch 270
Validation prc-auc = 0.428105
Epoch 271
Validation prc-auc = 0.427992
Epoch 272
Validation prc-auc = 0.427298
Epoch 273
Validation prc-auc = 0.421109
Epoch 274
Loss = 1.7398e-03, PNorm = 98.2798, GNorm = 0.1109, lr_0 = 6.3280e-04
Validation prc-auc = 0.412641
Epoch 275
Validation prc-auc = 0.409119
Epoch 276
Validation prc-auc = 0.405626
Epoch 277
Validation prc-auc = 0.409941
Epoch 278
Validation prc-auc = 0.409141
Epoch 279
Loss = 2.4879e-03, PNorm = 98.3120, GNorm = 0.0211, lr_0 = 6.2739e-04
Validation prc-auc = 0.412787
Epoch 280
Validation prc-auc = 0.413030
Epoch 281
Validation prc-auc = 0.412274
Epoch 282
Validation prc-auc = 0.417196
Epoch 283
Validation prc-auc = 0.418313
Epoch 284
Loss = 1.9315e-02, PNorm = 98.3444, GNorm = 1.1713, lr_0 = 6.2203e-04
Validation prc-auc = 0.426057
Epoch 285
Validation prc-auc = 0.435890
Epoch 286
Validation prc-auc = 0.434493
Epoch 287
Validation prc-auc = 0.434114
Epoch 288
Validation prc-auc = 0.433181
Epoch 289
Loss = 1.2084e-03, PNorm = 98.3772, GNorm = 0.0124, lr_0 = 6.1671e-04
Validation prc-auc = 0.424580
Epoch 290
Validation prc-auc = 0.423863
Epoch 291
Validation prc-auc = 0.423674
Epoch 292
Validation prc-auc = 0.423647
Epoch 293
Validation prc-auc = 0.423370
Epoch 294
Loss = 7.1318e-04, PNorm = 98.3994, GNorm = 0.0102, lr_0 = 6.1144e-04
Validation prc-auc = 0.423250
Epoch 295
Validation prc-auc = 0.423070
Epoch 296
Validation prc-auc = 0.423077
Epoch 297
Validation prc-auc = 0.422743
Epoch 298
Validation prc-auc = 0.430283
Epoch 299
Loss = 1.9712e-04, PNorm = 98.4124, GNorm = 0.0031, lr_0 = 6.0621e-04
Validation prc-auc = 0.430550
Model 0 best validation prc-auc = 0.478304 on epoch 66
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.213505
Ensemble test prc-auc = 0.213505
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.362497
Epoch 1
Validation prc-auc = 0.373475
Epoch 2
Validation prc-auc = 0.360164
Epoch 3
Validation prc-auc = 0.307196
Epoch 4
Loss = 5.7471e-01, PNorm = 87.5331, GNorm = 2.4418, lr_0 = 6.5000e-04
Validation prc-auc = 0.347058
Epoch 5
Validation prc-auc = 0.434601
Epoch 6
Validation prc-auc = 0.422351
Epoch 7
Validation prc-auc = 0.326135
Epoch 8
Validation prc-auc = 0.252556
Epoch 9
Loss = 6.0477e-01, PNorm = 87.8801, GNorm = 1.7058, lr_0 = 9.9743e-04
Validation prc-auc = 0.387881
Epoch 10
Validation prc-auc = 0.369805
Epoch 11
Validation prc-auc = 0.403910
Epoch 12
Validation prc-auc = 0.388718
Epoch 13
Validation prc-auc = 0.322996
Epoch 14
Loss = 4.4575e-01, PNorm = 88.2486, GNorm = 2.5967, lr_0 = 9.8890e-04
Validation prc-auc = 0.404555
Epoch 15
Validation prc-auc = 0.360549
Epoch 16
Validation prc-auc = 0.297851
Epoch 17
Validation prc-auc = 0.300639
Epoch 18
Validation prc-auc = 0.369419
Epoch 19
Loss = 3.9442e-01, PNorm = 88.5173, GNorm = 2.1465, lr_0 = 9.8045e-04
Validation prc-auc = 0.436479
Epoch 20
Validation prc-auc = 0.416071
Epoch 21
Validation prc-auc = 0.303731
Epoch 22
Validation prc-auc = 0.352473
Epoch 23
Validation prc-auc = 0.291816
Epoch 24
Loss = 4.3505e-01, PNorm = 88.7314, GNorm = 3.4450, lr_0 = 9.7207e-04
Validation prc-auc = 0.334164
Epoch 25
Validation prc-auc = 0.371015
Epoch 26
Validation prc-auc = 0.408142
Epoch 27
Validation prc-auc = 0.398162
Epoch 28
Validation prc-auc = 0.404370
Epoch 29
Loss = 3.2314e-01, PNorm = 88.9282, GNorm = 1.4325, lr_0 = 9.6376e-04
Validation prc-auc = 0.430300
Epoch 30
Validation prc-auc = 0.425676
Epoch 31
Validation prc-auc = 0.322605
Epoch 32
Validation prc-auc = 0.306415
Epoch 33
Validation prc-auc = 0.418706
Epoch 34
Loss = 1.6832e-01, PNorm = 89.1495, GNorm = 1.3843, lr_0 = 9.5552e-04
Validation prc-auc = 0.435868
Epoch 35
Validation prc-auc = 0.431030
Epoch 36
Validation prc-auc = 0.415866
Epoch 37
Validation prc-auc = 0.363612
Epoch 38
Validation prc-auc = 0.447383
Epoch 39
Loss = 2.0156e-01, PNorm = 89.3834, GNorm = 2.5952, lr_0 = 9.4735e-04
Validation prc-auc = 0.434065
Epoch 40
Validation prc-auc = 0.405605
Epoch 41
Validation prc-auc = 0.241574
Epoch 42
Validation prc-auc = 0.178179
Epoch 43
Validation prc-auc = 0.374672
Epoch 44
Loss = 3.4034e-01, PNorm = 89.6511, GNorm = 3.1377, lr_0 = 9.3925e-04
Validation prc-auc = 0.436223
Epoch 45
Validation prc-auc = 0.417430
Epoch 46
Validation prc-auc = 0.409178
Epoch 47
Validation prc-auc = 0.394049
Epoch 48
Validation prc-auc = 0.356961
Epoch 49
Loss = 2.4550e-01, PNorm = 89.9484, GNorm = 1.7773, lr_0 = 9.3122e-04
Validation prc-auc = 0.371936
Epoch 50
Validation prc-auc = 0.371113
Epoch 51
Validation prc-auc = 0.347611
Epoch 52
Validation prc-auc = 0.394731
Epoch 53
Validation prc-auc = 0.392802
Epoch 54
Loss = 1.7714e-01, PNorm = 90.2010, GNorm = 0.6216, lr_0 = 9.2326e-04
Validation prc-auc = 0.256179
Epoch 55
Validation prc-auc = 0.260006
Epoch 56
Validation prc-auc = 0.388468
Epoch 57
Validation prc-auc = 0.442836
Epoch 58
Validation prc-auc = 0.404542
Epoch 59
Loss = 1.7288e-01, PNorm = 90.4433, GNorm = 1.6112, lr_0 = 9.1537e-04
Validation prc-auc = 0.417048
Epoch 60
Validation prc-auc = 0.423662
Epoch 61
Validation prc-auc = 0.421955
Epoch 62
Validation prc-auc = 0.423658
Epoch 63
Validation prc-auc = 0.436347
Epoch 64
Loss = 1.3335e-01, PNorm = 90.7133, GNorm = 1.4831, lr_0 = 9.0754e-04
Validation prc-auc = 0.434111
Epoch 65
Validation prc-auc = 0.438346
Epoch 66
Validation prc-auc = 0.424796
Epoch 67
Validation prc-auc = 0.442682
Epoch 68
Validation prc-auc = 0.385831
Epoch 69
Loss = 1.1801e-01, PNorm = 90.9926, GNorm = 0.7664, lr_0 = 8.9978e-04
Validation prc-auc = 0.416065
Epoch 70
Validation prc-auc = 0.440929
Epoch 71
Validation prc-auc = 0.435454
Epoch 72
Validation prc-auc = 0.446181
Epoch 73
Validation prc-auc = 0.370424
Epoch 74
Loss = 1.9017e-01, PNorm = 91.2609, GNorm = 1.6306, lr_0 = 8.9209e-04
Validation prc-auc = 0.387250
Epoch 75
Validation prc-auc = 0.439166
Epoch 76
Validation prc-auc = 0.446470
Epoch 77
Validation prc-auc = 0.403975
Epoch 78
Validation prc-auc = 0.437590
Epoch 79
Loss = 1.1276e-01, PNorm = 91.5400, GNorm = 0.7200, lr_0 = 8.8447e-04
Validation prc-auc = 0.434295
Epoch 80
Validation prc-auc = 0.428552
Epoch 81
Validation prc-auc = 0.426195
Epoch 82
Validation prc-auc = 0.434644
Epoch 83
Validation prc-auc = 0.446133
Epoch 84
Loss = 4.1444e-02, PNorm = 91.8036, GNorm = 0.8984, lr_0 = 8.7691e-04
Validation prc-auc = 0.435470
Epoch 85
Validation prc-auc = 0.423392
Epoch 86
Validation prc-auc = 0.433673
Epoch 87
Validation prc-auc = 0.428869
Epoch 88
Validation prc-auc = 0.422552
Epoch 89
Loss = 1.2683e-01, PNorm = 92.0641, GNorm = 0.9304, lr_0 = 8.6941e-04
Validation prc-auc = 0.428762
Epoch 90
Validation prc-auc = 0.432585
Epoch 91
Validation prc-auc = 0.438298
Epoch 92
Validation prc-auc = 0.441559
Epoch 93
Validation prc-auc = 0.454075
Epoch 94
Loss = 1.0072e-01, PNorm = 92.3072, GNorm = 2.2394, lr_0 = 8.6198e-04
Validation prc-auc = 0.445418
Epoch 95
Validation prc-auc = 0.433576
Epoch 96
Validation prc-auc = 0.433155
Epoch 97
Validation prc-auc = 0.418477
Epoch 98
Validation prc-auc = 0.389189
Epoch 99
Loss = 1.3167e-01, PNorm = 92.5699, GNorm = 1.2070, lr_0 = 8.5461e-04
Validation prc-auc = 0.422612
Epoch 100
Validation prc-auc = 0.453752
Epoch 101
Validation prc-auc = 0.460648
Epoch 102
Validation prc-auc = 0.451273
Epoch 103
Validation prc-auc = 0.438268
Epoch 104
Loss = 2.7240e-01, PNorm = 92.9133, GNorm = 3.2094, lr_0 = 8.4730e-04
Validation prc-auc = 0.421892
Epoch 105
Validation prc-auc = 0.436371
Epoch 106
Validation prc-auc = 0.462895
Epoch 107
Validation prc-auc = 0.448141
Epoch 108
Validation prc-auc = 0.457891
Epoch 109
Loss = 6.4572e-02, PNorm = 93.2494, GNorm = 1.4651, lr_0 = 8.4006e-04
Validation prc-auc = 0.456054
Epoch 110
Validation prc-auc = 0.436480
Epoch 111
Validation prc-auc = 0.413980
Epoch 112
Validation prc-auc = 0.406138
Epoch 113
Validation prc-auc = 0.408682
Epoch 114
Loss = 1.0347e-01, PNorm = 93.5674, GNorm = 2.1408, lr_0 = 8.3288e-04
Validation prc-auc = 0.424562
Epoch 115
Validation prc-auc = 0.429416
Epoch 116
Validation prc-auc = 0.444542
Epoch 117
Validation prc-auc = 0.463586
Epoch 118
Validation prc-auc = 0.467807
Epoch 119
Loss = 9.7550e-02, PNorm = 93.8466, GNorm = 0.9734, lr_0 = 8.2576e-04
Validation prc-auc = 0.472326
Epoch 120
Validation prc-auc = 0.473393
Epoch 121
Validation prc-auc = 0.451979
Epoch 122
Validation prc-auc = 0.438334
Epoch 123
Validation prc-auc = 0.430406
Epoch 124
Loss = 6.2798e-02, PNorm = 94.0753, GNorm = 0.7937, lr_0 = 8.1870e-04
Validation prc-auc = 0.415055
Epoch 125
Validation prc-auc = 0.406324
Epoch 126
Validation prc-auc = 0.406993
Epoch 127
Validation prc-auc = 0.408127
Epoch 128
Validation prc-auc = 0.408767
Epoch 129
Loss = 2.3529e-02, PNorm = 94.2809, GNorm = 0.3728, lr_0 = 8.1170e-04
Validation prc-auc = 0.423886
Epoch 130
Validation prc-auc = 0.433273
Epoch 131
Validation prc-auc = 0.420602
Epoch 132
Validation prc-auc = 0.420445
Epoch 133
Validation prc-auc = 0.424049
Epoch 134
Loss = 5.1804e-02, PNorm = 94.4622, GNorm = 1.2415, lr_0 = 8.0476e-04
Validation prc-auc = 0.422727
Epoch 135
Validation prc-auc = 0.413270
Epoch 136
Validation prc-auc = 0.395384
Epoch 137
Validation prc-auc = 0.387073
Epoch 138
Validation prc-auc = 0.385723
Epoch 139
Loss = 1.7235e-02, PNorm = 94.6083, GNorm = 0.3096, lr_0 = 7.9788e-04
Validation prc-auc = 0.385808
Epoch 140
Validation prc-auc = 0.383456
Epoch 141
Validation prc-auc = 0.374238
Epoch 142
Validation prc-auc = 0.376041
Epoch 143
Validation prc-auc = 0.372745
Epoch 144
Loss = 1.9924e-02, PNorm = 94.7812, GNorm = 0.0584, lr_0 = 7.9106e-04
Validation prc-auc = 0.376354
Epoch 145
Validation prc-auc = 0.389142
Epoch 146
Validation prc-auc = 0.404956
Epoch 147
Validation prc-auc = 0.414389
Epoch 148
Validation prc-auc = 0.423214
Epoch 149
Loss = 5.6403e-02, PNorm = 94.9600, GNorm = 1.1076, lr_0 = 7.8430e-04
Validation prc-auc = 0.425686
Epoch 150
Validation prc-auc = 0.414375
Epoch 151
Validation prc-auc = 0.426917
Epoch 152
Validation prc-auc = 0.407678
Epoch 153
Validation prc-auc = 0.409563
Epoch 154
Loss = 2.5969e-02, PNorm = 95.1361, GNorm = 0.1092, lr_0 = 7.7759e-04
Validation prc-auc = 0.413258
Epoch 155
Validation prc-auc = 0.421704
Epoch 156
Validation prc-auc = 0.431467
Epoch 157
Validation prc-auc = 0.439384
Epoch 158
Validation prc-auc = 0.441315
Epoch 159
Loss = 4.5813e-02, PNorm = 95.2767, GNorm = 1.0817, lr_0 = 7.7095e-04
Validation prc-auc = 0.420635
Epoch 160
Validation prc-auc = 0.418439
Epoch 161
Validation prc-auc = 0.415463
Epoch 162
Validation prc-auc = 0.411232
Epoch 163
Validation prc-auc = 0.416592
Epoch 164
Loss = 3.7197e-02, PNorm = 95.3907, GNorm = 0.1878, lr_0 = 7.6436e-04
Validation prc-auc = 0.414037
Epoch 165
Validation prc-auc = 0.417224
Epoch 166
Validation prc-auc = 0.420740
Epoch 167
Validation prc-auc = 0.416032
Epoch 168
Validation prc-auc = 0.434841
Epoch 169
Loss = 4.8822e-02, PNorm = 95.4964, GNorm = 2.1253, lr_0 = 7.5782e-04
Validation prc-auc = 0.425846
Epoch 170
Validation prc-auc = 0.410523
Epoch 171
Validation prc-auc = 0.411176
Epoch 172
Validation prc-auc = 0.367697
Epoch 173
Validation prc-auc = 0.241142
Epoch 174
Loss = 1.2317e-01, PNorm = 95.7006, GNorm = 2.9845, lr_0 = 7.5134e-04
Validation prc-auc = 0.231739
Epoch 175
Validation prc-auc = 0.390899
Epoch 176
Validation prc-auc = 0.410981
Epoch 177
Validation prc-auc = 0.412194
Epoch 178
Validation prc-auc = 0.412781
Epoch 179
Loss = 6.0627e-02, PNorm = 96.0239, GNorm = 1.3169, lr_0 = 7.4492e-04
Validation prc-auc = 0.413172
Epoch 180
Validation prc-auc = 0.410910
Epoch 181
Validation prc-auc = 0.409262
Epoch 182
Validation prc-auc = 0.414819
Epoch 183
Validation prc-auc = 0.397468
Epoch 184
Loss = 3.5593e-02, PNorm = 96.3340, GNorm = 1.3012, lr_0 = 7.3855e-04
Validation prc-auc = 0.407506
Epoch 185
Validation prc-auc = 0.407030
Epoch 186
Validation prc-auc = 0.406498
Epoch 187
Validation prc-auc = 0.409389
Epoch 188
Validation prc-auc = 0.392049
Epoch 189
Loss = 2.0494e-02, PNorm = 96.5465, GNorm = 0.1019, lr_0 = 7.3224e-04
Validation prc-auc = 0.396802
Epoch 190
Validation prc-auc = 0.414088
Epoch 191
Validation prc-auc = 0.407545
Epoch 192
Validation prc-auc = 0.380882
Epoch 193
Validation prc-auc = 0.328893
Epoch 194
Loss = 1.2834e-02, PNorm = 96.7388, GNorm = 0.1016, lr_0 = 7.2598e-04
Validation prc-auc = 0.317114
Epoch 195
Validation prc-auc = 0.309646
Epoch 196
Validation prc-auc = 0.323808
Epoch 197
Validation prc-auc = 0.344727
Epoch 198
Validation prc-auc = 0.343460
Epoch 199
Loss = 1.3668e-02, PNorm = 96.9075, GNorm = 0.1764, lr_0 = 7.1977e-04
Validation prc-auc = 0.393597
Epoch 200
Validation prc-auc = 0.411827
Epoch 201
Validation prc-auc = 0.415010
Epoch 202
Validation prc-auc = 0.422301
Epoch 203
Validation prc-auc = 0.427784
Epoch 204
Loss = 1.6730e-02, PNorm = 97.0420, GNorm = 1.4057, lr_0 = 7.1362e-04
Validation prc-auc = 0.429437
Epoch 205
Validation prc-auc = 0.429440
Epoch 206
Validation prc-auc = 0.423601
Epoch 207
Validation prc-auc = 0.435355
Epoch 208
Validation prc-auc = 0.430451
Epoch 209
Loss = 2.1416e-02, PNorm = 97.1424, GNorm = 0.4492, lr_0 = 7.0752e-04
Validation prc-auc = 0.417132
Epoch 210
Validation prc-auc = 0.415782
Epoch 211
Validation prc-auc = 0.412640
Epoch 212
Validation prc-auc = 0.410880
Epoch 213
Validation prc-auc = 0.400444
Epoch 214
Loss = 1.3355e-02, PNorm = 97.2141, GNorm = 0.8666, lr_0 = 7.0147e-04
Validation prc-auc = 0.399334
Epoch 215
Validation prc-auc = 0.398111
Epoch 216
Validation prc-auc = 0.396740
Epoch 217
Validation prc-auc = 0.394147
Epoch 218
Validation prc-auc = 0.395745
Epoch 219
Loss = 5.3262e-02, PNorm = 97.2837, GNorm = 0.9150, lr_0 = 6.9548e-04
Validation prc-auc = 0.382216
Epoch 220
Validation prc-auc = 0.405865
Epoch 221
Validation prc-auc = 0.414806
Epoch 222
Validation prc-auc = 0.419688
Epoch 223
Validation prc-auc = 0.423608
Epoch 224
Loss = 2.7031e-02, PNorm = 97.3888, GNorm = 1.6029, lr_0 = 6.8953e-04
Validation prc-auc = 0.421314
Epoch 225
Validation prc-auc = 0.411181
Epoch 226
Validation prc-auc = 0.441492
Epoch 227
Validation prc-auc = 0.421565
Epoch 228
Validation prc-auc = 0.407030
Epoch 229
Loss = 1.1787e-02, PNorm = 97.5518, GNorm = 0.8230, lr_0 = 6.8364e-04
Validation prc-auc = 0.394626
Epoch 230
Validation prc-auc = 0.388724
Epoch 231
Validation prc-auc = 0.389461
Epoch 232
Validation prc-auc = 0.391337
Epoch 233
Validation prc-auc = 0.392381
Epoch 234
Loss = 4.3574e-02, PNorm = 97.6985, GNorm = 0.5728, lr_0 = 6.7779e-04
Validation prc-auc = 0.393538
Epoch 235
Validation prc-auc = 0.399377
Epoch 236
Validation prc-auc = 0.399375
Epoch 237
Validation prc-auc = 0.398904
Epoch 238
Validation prc-auc = 0.395708
Epoch 239
Loss = 7.3699e-03, PNorm = 97.8086, GNorm = 0.1509, lr_0 = 6.7200e-04
Validation prc-auc = 0.387274
Epoch 240
Validation prc-auc = 0.387553
Epoch 241
Validation prc-auc = 0.391090
Epoch 242
Validation prc-auc = 0.383828
Epoch 243
Validation prc-auc = 0.385722
Epoch 244
Loss = 1.3489e-02, PNorm = 97.9121, GNorm = 0.4102, lr_0 = 6.6625e-04
Validation prc-auc = 0.389365
Epoch 245
Validation prc-auc = 0.389433
Epoch 246
Validation prc-auc = 0.388472
Epoch 247
Validation prc-auc = 0.392304
Epoch 248
Validation prc-auc = 0.392445
Epoch 249
Loss = 7.1786e-03, PNorm = 98.0056, GNorm = 0.4079, lr_0 = 6.6056e-04
Validation prc-auc = 0.390365
Epoch 250
Validation prc-auc = 0.392715
Epoch 251
Validation prc-auc = 0.396124
Epoch 252
Validation prc-auc = 0.395942
Epoch 253
Validation prc-auc = 0.398262
Epoch 254
Loss = 1.1789e-03, PNorm = 98.0654, GNorm = 0.0238, lr_0 = 6.5491e-04
Validation prc-auc = 0.397996
Epoch 255
Validation prc-auc = 0.398191
Epoch 256
Validation prc-auc = 0.399006
Epoch 257
Validation prc-auc = 0.399429
Epoch 258
Validation prc-auc = 0.402382
Epoch 259
Loss = 4.9863e-02, PNorm = 98.1004, GNorm = 0.1003, lr_0 = 6.4931e-04
Validation prc-auc = 0.405264
Epoch 260
Validation prc-auc = 0.413860
Epoch 261
Validation prc-auc = 0.414864
Epoch 262
Validation prc-auc = 0.422074
Epoch 263
Validation prc-auc = 0.422394
Epoch 264
Loss = 1.5346e-03, PNorm = 98.1268, GNorm = 0.0674, lr_0 = 6.4376e-04
Validation prc-auc = 0.418219
Epoch 265
Validation prc-auc = 0.418300
Epoch 266
Validation prc-auc = 0.419142
Epoch 267
Validation prc-auc = 0.419063
Epoch 268
Validation prc-auc = 0.412502
Epoch 269
Loss = 2.5583e-03, PNorm = 98.1484, GNorm = 0.1399, lr_0 = 6.3826e-04
Validation prc-auc = 0.412200
Epoch 270
Validation prc-auc = 0.412412
Epoch 271
Validation prc-auc = 0.412261
Epoch 272
Validation prc-auc = 0.412046
Epoch 273
Validation prc-auc = 0.412046
Epoch 274
Loss = 6.6527e-04, PNorm = 98.1633, GNorm = 0.0293, lr_0 = 6.3280e-04
Validation prc-auc = 0.410055
Epoch 275
Validation prc-auc = 0.408544
Epoch 276
Validation prc-auc = 0.407576
Epoch 277
Validation prc-auc = 0.406944
Epoch 278
Validation prc-auc = 0.405887
Epoch 279
Loss = 6.7682e-04, PNorm = 98.1764, GNorm = 0.0230, lr_0 = 6.2739e-04
Validation prc-auc = 0.405664
Epoch 280
Validation prc-auc = 0.400857
Epoch 281
Validation prc-auc = 0.401464
Epoch 282
Validation prc-auc = 0.407146
Epoch 283
Validation prc-auc = 0.399679
Epoch 284
Loss = 2.3346e-02, PNorm = 98.1951, GNorm = 0.0352, lr_0 = 6.2203e-04
Validation prc-auc = 0.403089
Epoch 285
Validation prc-auc = 0.402942
Epoch 286
Validation prc-auc = 0.402453
Epoch 287
Validation prc-auc = 0.402943
Epoch 288
Validation prc-auc = 0.390514
Epoch 289
Loss = 3.0188e-02, PNorm = 98.2278, GNorm = 0.8437, lr_0 = 6.1671e-04
Validation prc-auc = 0.391393
Epoch 290
Validation prc-auc = 0.397193
Epoch 291
Validation prc-auc = 0.398494
Epoch 292
Validation prc-auc = 0.398624
Epoch 293
Validation prc-auc = 0.397816
Epoch 294
Loss = 3.9932e-03, PNorm = 98.2580, GNorm = 0.0248, lr_0 = 6.1144e-04
Validation prc-auc = 0.393736
Epoch 295
Validation prc-auc = 0.397001
Epoch 296
Validation prc-auc = 0.397505
Epoch 297
Validation prc-auc = 0.397523
Epoch 298
Validation prc-auc = 0.403771
Epoch 299
Loss = 6.4611e-04, PNorm = 98.2804, GNorm = 0.0266, lr_0 = 6.0621e-04
Validation prc-auc = 0.403617
Model 0 best validation prc-auc = 0.473393 on epoch 120
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.318417
Ensemble test prc-auc = 0.318417
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.364343
Epoch 1
Validation prc-auc = 0.378904
Epoch 2
Validation prc-auc = 0.410851
Epoch 3
Validation prc-auc = 0.383521
Epoch 4
Loss = 5.9631e-01, PNorm = 87.5360, GNorm = 2.1408, lr_0 = 6.5000e-04
Validation prc-auc = 0.391746
Epoch 5
Validation prc-auc = 0.383961
Epoch 6
Validation prc-auc = 0.281225
Epoch 7
Validation prc-auc = 0.286502
Epoch 8
Validation prc-auc = 0.360174
Epoch 9
Loss = 5.6768e-01, PNorm = 87.8905, GNorm = 3.8597, lr_0 = 9.9743e-04
Validation prc-auc = 0.384963
Epoch 10
Validation prc-auc = 0.349872
Epoch 11
Validation prc-auc = 0.221127
Epoch 12
Validation prc-auc = 0.340810
Epoch 13
Validation prc-auc = 0.398876
Epoch 14
Loss = 5.3910e-01, PNorm = 88.2642, GNorm = 2.5137, lr_0 = 9.8890e-04
Validation prc-auc = 0.326811
Epoch 15
Validation prc-auc = 0.203515
Epoch 16
Validation prc-auc = 0.257900
Epoch 17
Validation prc-auc = 0.411710
Epoch 18
Validation prc-auc = 0.334976
Epoch 19
Loss = 4.3846e-01, PNorm = 88.5578, GNorm = 2.2958, lr_0 = 9.8045e-04
Validation prc-auc = 0.342368
Epoch 20
Validation prc-auc = 0.373302
Epoch 21
Validation prc-auc = 0.390068
Epoch 22
Validation prc-auc = 0.348180
Epoch 23
Validation prc-auc = 0.444626
Epoch 24
Loss = 4.3891e-01, PNorm = 88.7966, GNorm = 1.8590, lr_0 = 9.7207e-04
Validation prc-auc = 0.419545
Epoch 25
Validation prc-auc = 0.415267
Epoch 26
Validation prc-auc = 0.402000
Epoch 27
Validation prc-auc = 0.343998
Epoch 28
Validation prc-auc = 0.247364
Epoch 29
Loss = 3.7850e-01, PNorm = 89.0080, GNorm = 1.9769, lr_0 = 9.6376e-04
Validation prc-auc = 0.349352
Epoch 30
Validation prc-auc = 0.421545
Epoch 31
Validation prc-auc = 0.414224
Epoch 32
Validation prc-auc = 0.321557
Epoch 33
Validation prc-auc = 0.309574
Epoch 34
Loss = 2.7933e-01, PNorm = 89.2314, GNorm = 1.9423, lr_0 = 9.5552e-04
Validation prc-auc = 0.402747
Epoch 35
Validation prc-auc = 0.437962
Epoch 36
Validation prc-auc = 0.413829
Epoch 37
Validation prc-auc = 0.375757
Epoch 38
Validation prc-auc = 0.384543
Epoch 39
Loss = 1.6271e-01, PNorm = 89.4585, GNorm = 1.1592, lr_0 = 9.4735e-04
Validation prc-auc = 0.405130
Epoch 40
Validation prc-auc = 0.423059
Epoch 41
Validation prc-auc = 0.378461
Epoch 42
Validation prc-auc = 0.373454
Epoch 43
Validation prc-auc = 0.307171
Epoch 44
Loss = 1.2338e-01, PNorm = 89.6956, GNorm = 1.1079, lr_0 = 9.3925e-04
Validation prc-auc = 0.413916
Epoch 45
Validation prc-auc = 0.429689
Epoch 46
Validation prc-auc = 0.434977
Epoch 47
Validation prc-auc = 0.404871
Epoch 48
Validation prc-auc = 0.436342
Epoch 49
Loss = 3.4135e-01, PNorm = 89.9547, GNorm = 2.3164, lr_0 = 9.3122e-04
Validation prc-auc = 0.428386
Epoch 50
Validation prc-auc = 0.425739
Epoch 51
Validation prc-auc = 0.390972
Epoch 52
Validation prc-auc = 0.406052
Epoch 53
Validation prc-auc = 0.439720
Epoch 54
Loss = 1.1110e-01, PNorm = 90.2702, GNorm = 0.8954, lr_0 = 9.2326e-04
Validation prc-auc = 0.430388
Epoch 55
Validation prc-auc = 0.430335
Epoch 56
Validation prc-auc = 0.437462
Epoch 57
Validation prc-auc = 0.422317
Epoch 58
Validation prc-auc = 0.446615
Epoch 59
Loss = 3.3311e-01, PNorm = 90.5466, GNorm = 3.3775, lr_0 = 9.1537e-04
Validation prc-auc = 0.422887
Epoch 60
Validation prc-auc = 0.394276
Epoch 61
Validation prc-auc = 0.387811
Epoch 62
Validation prc-auc = 0.397223
Epoch 63
Validation prc-auc = 0.406988
Epoch 64
Loss = 1.5634e-01, PNorm = 90.7714, GNorm = 1.1756, lr_0 = 9.0754e-04
Validation prc-auc = 0.408931
Epoch 65
Validation prc-auc = 0.412302
Epoch 66
Validation prc-auc = 0.412376
Epoch 67
Validation prc-auc = 0.417766
Epoch 68
Validation prc-auc = 0.419405
Epoch 69
Loss = 1.2612e-01, PNorm = 90.9616, GNorm = 1.2184, lr_0 = 8.9978e-04
Validation prc-auc = 0.404993
Epoch 70
Validation prc-auc = 0.387454
Epoch 71
Validation prc-auc = 0.250586
Epoch 72
Validation prc-auc = 0.239999
Epoch 73
Validation prc-auc = 0.397409
Epoch 74
Loss = 1.4739e-01, PNorm = 91.2126, GNorm = 1.0118, lr_0 = 8.9209e-04
Validation prc-auc = 0.424045
Epoch 75
Validation prc-auc = 0.442427
Epoch 76
Validation prc-auc = 0.435206
Epoch 77
Validation prc-auc = 0.428342
Epoch 78
Validation prc-auc = 0.427023
Epoch 79
Loss = 7.7433e-02, PNorm = 91.4929, GNorm = 0.9329, lr_0 = 8.8447e-04
Validation prc-auc = 0.428467
Epoch 80
Validation prc-auc = 0.432301
Epoch 81
Validation prc-auc = 0.414300
Epoch 82
Validation prc-auc = 0.412149
Epoch 83
Validation prc-auc = 0.417437
Epoch 84
Loss = 7.9287e-02, PNorm = 91.7761, GNorm = 0.8782, lr_0 = 8.7691e-04
Validation prc-auc = 0.412065
Epoch 85
Validation prc-auc = 0.387838
Epoch 86
Validation prc-auc = 0.395771
Epoch 87
Validation prc-auc = 0.401688
Epoch 88
Validation prc-auc = 0.443481
Epoch 89
Loss = 5.0597e-02, PNorm = 92.1119, GNorm = 0.9516, lr_0 = 8.6941e-04
Validation prc-auc = 0.445172
Epoch 90
Validation prc-auc = 0.442158
Epoch 91
Validation prc-auc = 0.439010
Epoch 92
Validation prc-auc = 0.434987
Epoch 93
Validation prc-auc = 0.429494
Epoch 94
Loss = 9.3143e-02, PNorm = 92.4654, GNorm = 1.4138, lr_0 = 8.6198e-04
Validation prc-auc = 0.405865
Epoch 95
Validation prc-auc = 0.421574
Epoch 96
Validation prc-auc = 0.418677
Epoch 97
Validation prc-auc = 0.428551
Epoch 98
Validation prc-auc = 0.439927
Epoch 99
Loss = 4.8138e-02, PNorm = 92.7287, GNorm = 0.7210, lr_0 = 8.5461e-04
Validation prc-auc = 0.463985
Epoch 100
Validation prc-auc = 0.425887
Epoch 101
Validation prc-auc = 0.429282
Epoch 102
Validation prc-auc = 0.414586
Epoch 103
Validation prc-auc = 0.432997
Epoch 104
Loss = 6.8647e-02, PNorm = 92.9366, GNorm = 1.8628, lr_0 = 8.4730e-04
Validation prc-auc = 0.425988
Epoch 105
Validation prc-auc = 0.401484
Epoch 106
Validation prc-auc = 0.398282
Epoch 107
Validation prc-auc = 0.408156
Epoch 108
Validation prc-auc = 0.406866
Epoch 109
Loss = 3.5722e-02, PNorm = 93.1200, GNorm = 0.7500, lr_0 = 8.4006e-04
Validation prc-auc = 0.420289
Epoch 110
Validation prc-auc = 0.422936
Epoch 111
Validation prc-auc = 0.411647
Epoch 112
Validation prc-auc = 0.398547
Epoch 113
Validation prc-auc = 0.380262
Epoch 114
Loss = 9.8076e-02, PNorm = 93.3025, GNorm = 1.1218, lr_0 = 8.3288e-04
Validation prc-auc = 0.414393
Epoch 115
Validation prc-auc = 0.424290
Epoch 116
Validation prc-auc = 0.418231
Epoch 117
Validation prc-auc = 0.418152
Epoch 118
Validation prc-auc = 0.383692
Epoch 119
Loss = 1.6673e-01, PNorm = 93.5183, GNorm = 1.6729, lr_0 = 8.2576e-04
Validation prc-auc = 0.286161
Epoch 120
Validation prc-auc = 0.299756
Epoch 121
Validation prc-auc = 0.430183
Epoch 122
Validation prc-auc = 0.439684
Epoch 123
Validation prc-auc = 0.445955
Epoch 124
Loss = 4.8920e-02, PNorm = 93.7324, GNorm = 0.5987, lr_0 = 8.1870e-04
Validation prc-auc = 0.435389
Epoch 125
Validation prc-auc = 0.431269
Epoch 126
Validation prc-auc = 0.420796
Epoch 127
Validation prc-auc = 0.436170
Epoch 128
Validation prc-auc = 0.421000
Epoch 129
Loss = 3.2339e-02, PNorm = 93.9089, GNorm = 0.4130, lr_0 = 8.1170e-04
Validation prc-auc = 0.397575
Epoch 130
Validation prc-auc = 0.392491
Epoch 131
Validation prc-auc = 0.408740
Epoch 132
Validation prc-auc = 0.430837
Epoch 133
Validation prc-auc = 0.428131
Epoch 134
Loss = 4.5445e-02, PNorm = 94.0640, GNorm = 0.2851, lr_0 = 8.0476e-04
Validation prc-auc = 0.446676
Epoch 135
Validation prc-auc = 0.439671
Epoch 136
Validation prc-auc = 0.430922
Epoch 137
Validation prc-auc = 0.426061
Epoch 138
Validation prc-auc = 0.423717
Epoch 139
Loss = 1.4727e-02, PNorm = 94.2060, GNorm = 0.5567, lr_0 = 7.9788e-04
Validation prc-auc = 0.422357
Epoch 140
Validation prc-auc = 0.434278
Epoch 141
Validation prc-auc = 0.440890
Epoch 142
Validation prc-auc = 0.416455
Epoch 143
Validation prc-auc = 0.416979
Epoch 144
Loss = 9.9377e-02, PNorm = 94.3543, GNorm = 0.3241, lr_0 = 7.9106e-04
Validation prc-auc = 0.435594
Epoch 145
Validation prc-auc = 0.439446
Epoch 146
Validation prc-auc = 0.443617
Epoch 147
Validation prc-auc = 0.444427
Epoch 148
Validation prc-auc = 0.426821
Epoch 149
Loss = 4.7719e-02, PNorm = 94.5646, GNorm = 1.2816, lr_0 = 7.8430e-04
Validation prc-auc = 0.425348
Epoch 150
Validation prc-auc = 0.410393
Epoch 151
Validation prc-auc = 0.396867
Epoch 152
Validation prc-auc = 0.391578
Epoch 153
Validation prc-auc = 0.413226
Epoch 154
Loss = 9.3455e-03, PNorm = 94.7305, GNorm = 0.0607, lr_0 = 7.7759e-04
Validation prc-auc = 0.412052
Epoch 155
Validation prc-auc = 0.421545
Epoch 156
Validation prc-auc = 0.418136
Epoch 157
Validation prc-auc = 0.419925
Epoch 158
Validation prc-auc = 0.436395
Epoch 159
Loss = 3.7724e-02, PNorm = 94.8429, GNorm = 0.1607, lr_0 = 7.7095e-04
Validation prc-auc = 0.423777
Epoch 160
Validation prc-auc = 0.432037
Epoch 161
Validation prc-auc = 0.432977
Epoch 162
Validation prc-auc = 0.426463
Epoch 163
Validation prc-auc = 0.429302
Epoch 164
Loss = 5.6646e-02, PNorm = 94.9364, GNorm = 1.1662, lr_0 = 7.6436e-04
Validation prc-auc = 0.433807
Epoch 165
Validation prc-auc = 0.427093
Epoch 166
Validation prc-auc = 0.427332
Epoch 167
Validation prc-auc = 0.433555
Epoch 168
Validation prc-auc = 0.444753
Epoch 169
Loss = 2.7726e-02, PNorm = 95.0668, GNorm = 0.9058, lr_0 = 7.5782e-04
Validation prc-auc = 0.437518
Epoch 170
Validation prc-auc = 0.444029
Epoch 171
Validation prc-auc = 0.444219
Epoch 172
Validation prc-auc = 0.445366
Epoch 173
Validation prc-auc = 0.439628
Epoch 174
Loss = 4.5124e-02, PNorm = 95.2081, GNorm = 0.1913, lr_0 = 7.5134e-04
Validation prc-auc = 0.440415
Epoch 175
Validation prc-auc = 0.440916
Epoch 176
Validation prc-auc = 0.443773
Epoch 177
Validation prc-auc = 0.448860
Epoch 178
Validation prc-auc = 0.448482
Epoch 179
Loss = 8.2944e-02, PNorm = 95.3518, GNorm = 0.4843, lr_0 = 7.4492e-04
Validation prc-auc = 0.446678
Epoch 180
Validation prc-auc = 0.440384
Epoch 181
Validation prc-auc = 0.437229
Epoch 182
Validation prc-auc = 0.434185
Epoch 183
Validation prc-auc = 0.431893
Epoch 184
Loss = 4.0652e-02, PNorm = 95.4841, GNorm = 0.1808, lr_0 = 7.3855e-04
Validation prc-auc = 0.414484
Epoch 185
Validation prc-auc = 0.417940
Epoch 186
Validation prc-auc = 0.427782
Epoch 187
Validation prc-auc = 0.441101
Epoch 188
Validation prc-auc = 0.443138
Epoch 189
Loss = 1.0636e-01, PNorm = 95.6123, GNorm = 2.1709, lr_0 = 7.3224e-04
Validation prc-auc = 0.440660
Epoch 190
Validation prc-auc = 0.438966
Epoch 191
Validation prc-auc = 0.431485
Epoch 192
Validation prc-auc = 0.432266
Epoch 193
Validation prc-auc = 0.431610
Epoch 194
Loss = 2.7890e-02, PNorm = 95.7485, GNorm = 0.8738, lr_0 = 7.2598e-04
Validation prc-auc = 0.430775
Epoch 195
Validation prc-auc = 0.430505
Epoch 196
Validation prc-auc = 0.429012
Epoch 197
Validation prc-auc = 0.428103
Epoch 198
Validation prc-auc = 0.426136
Epoch 199
Loss = 2.2492e-03, PNorm = 95.8536, GNorm = 0.0593, lr_0 = 7.1977e-04
Validation prc-auc = 0.420880
Epoch 200
Validation prc-auc = 0.424363
Epoch 201
Validation prc-auc = 0.422725
Epoch 202
Validation prc-auc = 0.420100
Epoch 203
Validation prc-auc = 0.424568
Epoch 204
Loss = 1.3250e-02, PNorm = 95.9360, GNorm = 0.6099, lr_0 = 7.1362e-04
Validation prc-auc = 0.434319
Epoch 205
Validation prc-auc = 0.435574
Epoch 206
Validation prc-auc = 0.418555
Epoch 207
Validation prc-auc = 0.412803
Epoch 208
Validation prc-auc = 0.407842
Epoch 209
Loss = 5.3621e-02, PNorm = 96.0309, GNorm = 0.0127, lr_0 = 7.0752e-04
Validation prc-auc = 0.419214
Epoch 210
Validation prc-auc = 0.438328
Epoch 211
Validation prc-auc = 0.417660
Epoch 212
Validation prc-auc = 0.403539
Epoch 213
Validation prc-auc = 0.401383
Epoch 214
Loss = 6.8434e-02, PNorm = 96.1335, GNorm = 0.8932, lr_0 = 7.0147e-04
Validation prc-auc = 0.397344
Epoch 215
Validation prc-auc = 0.395815
Epoch 216
Validation prc-auc = 0.411607
Epoch 217
Validation prc-auc = 0.417362
Epoch 218
Validation prc-auc = 0.412202
Epoch 219
Loss = 1.8518e-02, PNorm = 96.2490, GNorm = 0.5765, lr_0 = 6.9548e-04
Validation prc-auc = 0.420528
Epoch 220
Validation prc-auc = 0.423060
Epoch 221
Validation prc-auc = 0.424916
Epoch 222
Validation prc-auc = 0.421799
Epoch 223
Validation prc-auc = 0.417622
Epoch 224
Loss = 1.5868e-02, PNorm = 96.3732, GNorm = 0.9355, lr_0 = 6.8953e-04
Validation prc-auc = 0.432484
Epoch 225
Validation prc-auc = 0.428821
Epoch 226
Validation prc-auc = 0.430027
Epoch 227
Validation prc-auc = 0.432317
Epoch 228
Validation prc-auc = 0.432750
Epoch 229
Loss = 8.1381e-03, PNorm = 96.4826, GNorm = 0.1106, lr_0 = 6.8364e-04
Validation prc-auc = 0.426498
Epoch 230
Validation prc-auc = 0.420102
Epoch 231
Validation prc-auc = 0.419512
Epoch 232
Validation prc-auc = 0.419315
Epoch 233
Validation prc-auc = 0.425039
Epoch 234
Loss = 9.2303e-03, PNorm = 96.5738, GNorm = 0.3965, lr_0 = 6.7779e-04
Validation prc-auc = 0.418180
Epoch 235
Validation prc-auc = 0.426796
Epoch 236
Validation prc-auc = 0.426136
Epoch 237
Validation prc-auc = 0.425749
Epoch 238
Validation prc-auc = 0.425651
Epoch 239
Loss = 4.7591e-03, PNorm = 96.6452, GNorm = 0.2372, lr_0 = 6.7200e-04
Validation prc-auc = 0.425976
Epoch 240
Validation prc-auc = 0.425213
Epoch 241
Validation prc-auc = 0.425646
Epoch 242
Validation prc-auc = 0.425740
Epoch 243
Validation prc-auc = 0.425430
Epoch 244
Loss = 1.6130e-03, PNorm = 96.6920, GNorm = 0.0528, lr_0 = 6.6625e-04
Validation prc-auc = 0.432842
Epoch 245
Validation prc-auc = 0.433812
Epoch 246
Validation prc-auc = 0.434581
Epoch 247
Validation prc-auc = 0.434640
Epoch 248
Validation prc-auc = 0.434678
Epoch 249
Loss = 1.7950e-02, PNorm = 96.7317, GNorm = 1.7391, lr_0 = 6.6056e-04
Validation prc-auc = 0.434445
Epoch 250
Validation prc-auc = 0.433083
Epoch 251
Validation prc-auc = 0.433422
Epoch 252
Validation prc-auc = 0.408012
Epoch 253
Validation prc-auc = 0.406903
Epoch 254
Loss = 4.1825e-03, PNorm = 96.7866, GNorm = 0.4062, lr_0 = 6.5491e-04
Validation prc-auc = 0.405811
Epoch 255
Validation prc-auc = 0.387618
Epoch 256
Validation prc-auc = 0.389550
Epoch 257
Validation prc-auc = 0.413208
Epoch 258
Validation prc-auc = 0.416516
Epoch 259
Loss = 1.1984e-02, PNorm = 96.8435, GNorm = 0.6341, lr_0 = 6.4931e-04
Validation prc-auc = 0.418628
Epoch 260
Validation prc-auc = 0.419738
Epoch 261
Validation prc-auc = 0.418851
Epoch 262
Validation prc-auc = 0.411441
Epoch 263
Validation prc-auc = 0.404810
Epoch 264
Loss = 3.1516e-04, PNorm = 96.9062, GNorm = 0.0089, lr_0 = 6.4376e-04
Validation prc-auc = 0.385312
Epoch 265
Validation prc-auc = 0.403241
Epoch 266
Validation prc-auc = 0.398338
Epoch 267
Validation prc-auc = 0.405040
Epoch 268
Validation prc-auc = 0.416641
Epoch 269
Loss = 1.1493e-03, PNorm = 96.9698, GNorm = 0.0217, lr_0 = 6.3826e-04
Validation prc-auc = 0.427473
Epoch 270
Validation prc-auc = 0.428652
Epoch 271
Validation prc-auc = 0.440928
Epoch 272
Validation prc-auc = 0.430891
Epoch 273
Validation prc-auc = 0.423232
Epoch 274
Loss = 2.4410e-03, PNorm = 97.0374, GNorm = 0.1173, lr_0 = 6.3280e-04
Validation prc-auc = 0.422299
Epoch 275
Validation prc-auc = 0.412604
Epoch 276
Validation prc-auc = 0.409414
Epoch 277
Validation prc-auc = 0.408460
Epoch 278
Validation prc-auc = 0.407756
Epoch 279
Loss = 5.5723e-02, PNorm = 97.1051, GNorm = 0.9950, lr_0 = 6.2739e-04
Validation prc-auc = 0.440890
Epoch 280
Validation prc-auc = 0.448241
Epoch 281
Validation prc-auc = 0.420948
Epoch 282
Validation prc-auc = 0.420898
Epoch 283
Validation prc-auc = 0.432935
Epoch 284
Loss = 2.1199e-02, PNorm = 97.2226, GNorm = 0.0328, lr_0 = 6.2203e-04
Validation prc-auc = 0.431978
Epoch 285
Validation prc-auc = 0.429568
Epoch 286
Validation prc-auc = 0.417614
Epoch 287
Validation prc-auc = 0.416193
Epoch 288
Validation prc-auc = 0.414515
Epoch 289
Loss = 3.4259e-02, PNorm = 97.3633, GNorm = 1.4673, lr_0 = 6.1671e-04
Validation prc-auc = 0.421778
Epoch 290
Validation prc-auc = 0.416251
Epoch 291
Validation prc-auc = 0.418203
Epoch 292
Validation prc-auc = 0.410650
Epoch 293
Validation prc-auc = 0.379249
Epoch 294
Loss = 4.0977e-02, PNorm = 97.5754, GNorm = 1.4586, lr_0 = 6.1144e-04
Validation prc-auc = 0.373291
Epoch 295
Validation prc-auc = 0.368327
Epoch 296
Validation prc-auc = 0.371613
Epoch 297
Validation prc-auc = 0.373478
Epoch 298
Validation prc-auc = 0.386336
Epoch 299
Loss = 1.1696e-02, PNorm = 97.7567, GNorm = 0.8590, lr_0 = 6.0621e-04
Validation prc-auc = 0.386250
Model 0 best validation prc-auc = 0.463985 on epoch 99
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.362098
Ensemble test prc-auc = 0.362098
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.414257
Epoch 1
Validation prc-auc = 0.445717
Epoch 2
Validation prc-auc = 0.365733
Epoch 3
Validation prc-auc = 0.444811
Epoch 4
Loss = 4.8792e-01, PNorm = 87.5371, GNorm = 1.7764, lr_0 = 6.5000e-04
Validation prc-auc = 0.415945
Epoch 5
Validation prc-auc = 0.376035
Epoch 6
Validation prc-auc = 0.293517
Epoch 7
Validation prc-auc = 0.387482
Epoch 8
Validation prc-auc = 0.394404
Epoch 9
Loss = 5.0587e-01, PNorm = 87.8806, GNorm = 2.1501, lr_0 = 9.9743e-04
Validation prc-auc = 0.263978
Epoch 10
Validation prc-auc = 0.346380
Epoch 11
Validation prc-auc = 0.404720
Epoch 12
Validation prc-auc = 0.262759
Epoch 13
Validation prc-auc = 0.385108
Epoch 14
Loss = 4.5688e-01, PNorm = 88.2497, GNorm = 2.3922, lr_0 = 9.8890e-04
Validation prc-auc = 0.408951
Epoch 15
Validation prc-auc = 0.237942
Epoch 16
Validation prc-auc = 0.259056
Epoch 17
Validation prc-auc = 0.259097
Epoch 18
Validation prc-auc = 0.280541
Epoch 19
Loss = 4.3348e-01, PNorm = 88.5273, GNorm = 1.8194, lr_0 = 9.8045e-04
Validation prc-auc = 0.260061
Epoch 20
Validation prc-auc = 0.295962
Epoch 21
Validation prc-auc = 0.347240
Epoch 22
Validation prc-auc = 0.308684
Epoch 23
Validation prc-auc = 0.338370
Epoch 24
Loss = 3.2698e-01, PNorm = 88.7609, GNorm = 1.7706, lr_0 = 9.7207e-04
Validation prc-auc = 0.253040
Epoch 25
Validation prc-auc = 0.400463
Epoch 26
Validation prc-auc = 0.437469
Epoch 27
Validation prc-auc = 0.414158
Epoch 28
Validation prc-auc = 0.348757
Epoch 29
Loss = 2.1959e-01, PNorm = 88.9861, GNorm = 1.1000, lr_0 = 9.6376e-04
Validation prc-auc = 0.260845
Epoch 30
Validation prc-auc = 0.249586
Epoch 31
Validation prc-auc = 0.258141
Epoch 32
Validation prc-auc = 0.378642
Epoch 33
Validation prc-auc = 0.398597
Epoch 34
Loss = 2.7667e-01, PNorm = 89.2060, GNorm = 2.3716, lr_0 = 9.5552e-04
Validation prc-auc = 0.389978
Epoch 35
Validation prc-auc = 0.320921
Epoch 36
Validation prc-auc = 0.342847
Epoch 37
Validation prc-auc = 0.194324
Epoch 38
Validation prc-auc = 0.202048
Epoch 39
Loss = 4.9899e-01, PNorm = 89.4473, GNorm = 4.3963, lr_0 = 9.4735e-04
Validation prc-auc = 0.341203
Epoch 40
Validation prc-auc = 0.410476
Epoch 41
Validation prc-auc = 0.432449
Epoch 42
Validation prc-auc = 0.386718
Epoch 43
Validation prc-auc = 0.413776
Epoch 44
Loss = 2.8382e-01, PNorm = 89.7025, GNorm = 2.4755, lr_0 = 9.3925e-04
Validation prc-auc = 0.395525
Epoch 45
Validation prc-auc = 0.383990
Epoch 46
Validation prc-auc = 0.378716
Epoch 47
Validation prc-auc = 0.431232
Epoch 48
Validation prc-auc = 0.430528
Epoch 49
Loss = 1.3218e-01, PNorm = 89.9337, GNorm = 1.3003, lr_0 = 9.3122e-04
Validation prc-auc = 0.449808
Epoch 50
Validation prc-auc = 0.455624
Epoch 51
Validation prc-auc = 0.444308
Epoch 52
Validation prc-auc = 0.432561
Epoch 53
Validation prc-auc = 0.393700
Epoch 54
Loss = 1.9824e-01, PNorm = 90.1858, GNorm = 1.2996, lr_0 = 9.2326e-04
Validation prc-auc = 0.432853
Epoch 55
Validation prc-auc = 0.445734
Epoch 56
Validation prc-auc = 0.430223
Epoch 57
Validation prc-auc = 0.424888
Epoch 58
Validation prc-auc = 0.434187
Epoch 59
Loss = 1.0130e-01, PNorm = 90.4333, GNorm = 0.5935, lr_0 = 9.1537e-04
Validation prc-auc = 0.371624
Epoch 60
Validation prc-auc = 0.346728
Epoch 61
Validation prc-auc = 0.414784
Epoch 62
Validation prc-auc = 0.405778
Epoch 63
Validation prc-auc = 0.420758
Epoch 64
Loss = 2.0609e-01, PNorm = 90.6722, GNorm = 1.4302, lr_0 = 9.0754e-04
Validation prc-auc = 0.398335
Epoch 65
Validation prc-auc = 0.354151
Epoch 66
Validation prc-auc = 0.413318
Epoch 67
Validation prc-auc = 0.422843
Epoch 68
Validation prc-auc = 0.437707
Epoch 69
Loss = 1.5677e-01, PNorm = 90.9076, GNorm = 1.0482, lr_0 = 8.9978e-04
Validation prc-auc = 0.431118
Epoch 70
Validation prc-auc = 0.392382
Epoch 71
Validation prc-auc = 0.351569
Epoch 72
Validation prc-auc = 0.369591
Epoch 73
Validation prc-auc = 0.395721
Epoch 74
Loss = 1.0241e-01, PNorm = 91.1485, GNorm = 0.5349, lr_0 = 8.9209e-04
Validation prc-auc = 0.404708
Epoch 75
Validation prc-auc = 0.405028
Epoch 76
Validation prc-auc = 0.406141
Epoch 77
Validation prc-auc = 0.412404
Epoch 78
Validation prc-auc = 0.447381
Epoch 79
Loss = 4.4910e-02, PNorm = 91.3957, GNorm = 1.0633, lr_0 = 8.8447e-04
Validation prc-auc = 0.444434
Epoch 80
Validation prc-auc = 0.445214
Epoch 81
Validation prc-auc = 0.441746
Epoch 82
Validation prc-auc = 0.428528
Epoch 83
Validation prc-auc = 0.403971
Epoch 84
Loss = 5.6289e-02, PNorm = 91.6523, GNorm = 1.0589, lr_0 = 8.7691e-04
Validation prc-auc = 0.373547
Epoch 85
Validation prc-auc = 0.325999
Epoch 86
Validation prc-auc = 0.367043
Epoch 87
Validation prc-auc = 0.413119
Epoch 88
Validation prc-auc = 0.416627
Epoch 89
Loss = 9.0283e-02, PNorm = 91.9474, GNorm = 1.5438, lr_0 = 8.6941e-04
Validation prc-auc = 0.424449
Epoch 90
Validation prc-auc = 0.434062
Epoch 91
Validation prc-auc = 0.427110
Epoch 92
Validation prc-auc = 0.439104
Epoch 93
Validation prc-auc = 0.452766
Epoch 94
Loss = 9.1407e-02, PNorm = 92.2560, GNorm = 1.0202, lr_0 = 8.6198e-04
Validation prc-auc = 0.441180
Epoch 95
Validation prc-auc = 0.432422
Epoch 96
Validation prc-auc = 0.423221
Epoch 97
Validation prc-auc = 0.426981
Epoch 98
Validation prc-auc = 0.443946
Epoch 99
Loss = 2.6322e-02, PNorm = 92.5606, GNorm = 0.5497, lr_0 = 8.5461e-04
Validation prc-auc = 0.439767
Epoch 100
Validation prc-auc = 0.442779
Epoch 101
Validation prc-auc = 0.447234
Epoch 102
Validation prc-auc = 0.428686
Epoch 103
Validation prc-auc = 0.424161
Epoch 104
Loss = 4.9458e-02, PNorm = 92.8435, GNorm = 0.5047, lr_0 = 8.4730e-04
Validation prc-auc = 0.394878
Epoch 105
Validation prc-auc = 0.372891
Epoch 106
Validation prc-auc = 0.393554
Epoch 107
Validation prc-auc = 0.418371
Epoch 108
Validation prc-auc = 0.425239
Epoch 109
Loss = 3.0487e-02, PNorm = 93.0895, GNorm = 0.8173, lr_0 = 8.4006e-04
Validation prc-auc = 0.425300
Epoch 110
Validation prc-auc = 0.430988
Epoch 111
Validation prc-auc = 0.440904
Epoch 112
Validation prc-auc = 0.456754
Epoch 113
Validation prc-auc = 0.453794
Epoch 114
Loss = 1.5223e-01, PNorm = 93.3241, GNorm = 3.8167, lr_0 = 8.3288e-04
Validation prc-auc = 0.469806
Epoch 115
Validation prc-auc = 0.443031
Epoch 116
Validation prc-auc = 0.442664
Epoch 117
Validation prc-auc = 0.442610
Epoch 118
Validation prc-auc = 0.430536
Epoch 119
Loss = 4.8094e-02, PNorm = 93.5464, GNorm = 0.6598, lr_0 = 8.2576e-04
Validation prc-auc = 0.422791
Epoch 120
Validation prc-auc = 0.419911
Epoch 121
Validation prc-auc = 0.431982
Epoch 122
Validation prc-auc = 0.411242
Epoch 123
Validation prc-auc = 0.412447
Epoch 124
Loss = 2.8191e-02, PNorm = 93.7383, GNorm = 0.3049, lr_0 = 8.1870e-04
Validation prc-auc = 0.429503
Epoch 125
Validation prc-auc = 0.430787
Epoch 126
Validation prc-auc = 0.431533
Epoch 127
Validation prc-auc = 0.425769
Epoch 128
Validation prc-auc = 0.420926
Epoch 129
Loss = 8.1460e-02, PNorm = 93.9105, GNorm = 1.3504, lr_0 = 8.1170e-04
Validation prc-auc = 0.413367
Epoch 130
Validation prc-auc = 0.404056
Epoch 131
Validation prc-auc = 0.402591
Epoch 132
Validation prc-auc = 0.422768
Epoch 133
Validation prc-auc = 0.427689
Epoch 134
Loss = 1.2406e-01, PNorm = 94.1227, GNorm = 1.5031, lr_0 = 8.0476e-04
Validation prc-auc = 0.416612
Epoch 135
Validation prc-auc = 0.419956
Epoch 136
Validation prc-auc = 0.458311
Epoch 137
Validation prc-auc = 0.438103
Epoch 138
Validation prc-auc = 0.432592
Epoch 139
Loss = 1.6509e-02, PNorm = 94.3390, GNorm = 0.7645, lr_0 = 7.9788e-04
Validation prc-auc = 0.430015
Epoch 140
Validation prc-auc = 0.430373
Epoch 141
Validation prc-auc = 0.429476
Epoch 142
Validation prc-auc = 0.436092
Epoch 143
Validation prc-auc = 0.445442
Epoch 144
Loss = 4.9223e-02, PNorm = 94.5784, GNorm = 0.8949, lr_0 = 7.9106e-04
Validation prc-auc = 0.458938
Epoch 145
Validation prc-auc = 0.443331
Epoch 146
Validation prc-auc = 0.445399
Epoch 147
Validation prc-auc = 0.446234
Epoch 148
Validation prc-auc = 0.463972
Epoch 149
Loss = 9.5756e-02, PNorm = 94.8100, GNorm = 2.1245, lr_0 = 7.8430e-04
Validation prc-auc = 0.453253
Epoch 150
Validation prc-auc = 0.464067
Epoch 151
Validation prc-auc = 0.461694
Epoch 152
Validation prc-auc = 0.466315
Epoch 153
Validation prc-auc = 0.466976
Epoch 154
Loss = 1.8820e-02, PNorm = 95.0046, GNorm = 0.1107, lr_0 = 7.7759e-04
Validation prc-auc = 0.466458
Epoch 155
Validation prc-auc = 0.468165
Epoch 156
Validation prc-auc = 0.470489
Epoch 157
Validation prc-auc = 0.461181
Epoch 158
Validation prc-auc = 0.449548
Epoch 159
Loss = 8.3060e-03, PNorm = 95.1600, GNorm = 0.0952, lr_0 = 7.7095e-04
Validation prc-auc = 0.448867
Epoch 160
Validation prc-auc = 0.448314
Epoch 161
Validation prc-auc = 0.447612
Epoch 162
Validation prc-auc = 0.446956
Epoch 163
Validation prc-auc = 0.449003
Epoch 164
Loss = 2.1396e-02, PNorm = 95.2768, GNorm = 1.1825, lr_0 = 7.6436e-04
Validation prc-auc = 0.463080
Epoch 165
Validation prc-auc = 0.456340
Epoch 166
Validation prc-auc = 0.463761
Epoch 167
Validation prc-auc = 0.448217
Epoch 168
Validation prc-auc = 0.449100
Epoch 169
Loss = 4.9767e-03, PNorm = 95.4083, GNorm = 0.1102, lr_0 = 7.5782e-04
Validation prc-auc = 0.449182
Epoch 170
Validation prc-auc = 0.450584
Epoch 171
Validation prc-auc = 0.451586
Epoch 172
Validation prc-auc = 0.452685
Epoch 173
Validation prc-auc = 0.449846
Epoch 174
Loss = 2.4959e-02, PNorm = 95.5133, GNorm = 1.2776, lr_0 = 7.5134e-04
Validation prc-auc = 0.449279
Epoch 175
Validation prc-auc = 0.439496
Epoch 176
Validation prc-auc = 0.443981
Epoch 177
Validation prc-auc = 0.441916
Epoch 178
Validation prc-auc = 0.440199
Epoch 179
Loss = 3.1570e-03, PNorm = 95.6031, GNorm = 0.0375, lr_0 = 7.4492e-04
Validation prc-auc = 0.438449
Epoch 180
Validation prc-auc = 0.435599
Epoch 181
Validation prc-auc = 0.421923
Epoch 182
Validation prc-auc = 0.410207
Epoch 183
Validation prc-auc = 0.421532
Epoch 184
Loss = 6.5129e-02, PNorm = 95.7009, GNorm = 0.0546, lr_0 = 7.3855e-04
Validation prc-auc = 0.429442
Epoch 185
Validation prc-auc = 0.435050
Epoch 186
Validation prc-auc = 0.442430
Epoch 187
Validation prc-auc = 0.450240
Epoch 188
Validation prc-auc = 0.441756
Epoch 189
Loss = 2.9428e-03, PNorm = 95.8249, GNorm = 0.2305, lr_0 = 7.3224e-04
Validation prc-auc = 0.453823
Epoch 190
Validation prc-auc = 0.440253
Epoch 191
Validation prc-auc = 0.433162
Epoch 192
Validation prc-auc = 0.436912
Epoch 193
Validation prc-auc = 0.436406
Epoch 194
Loss = 1.3222e-02, PNorm = 95.9294, GNorm = 0.7511, lr_0 = 7.2598e-04
Validation prc-auc = 0.438686
Epoch 195
Validation prc-auc = 0.442675
Epoch 196
Validation prc-auc = 0.457430
Epoch 197
Validation prc-auc = 0.458213
Epoch 198
Validation prc-auc = 0.458897
Epoch 199
Loss = 4.5268e-03, PNorm = 96.0207, GNorm = 0.2626, lr_0 = 7.1977e-04
Validation prc-auc = 0.458597
Epoch 200
Validation prc-auc = 0.455517
Epoch 201
Validation prc-auc = 0.459790
Epoch 202
Validation prc-auc = 0.457319
Epoch 203
Validation prc-auc = 0.454542
Epoch 204
Loss = 4.5493e-02, PNorm = 96.1463, GNorm = 0.7230, lr_0 = 7.1362e-04
Validation prc-auc = 0.442614
Epoch 205
Validation prc-auc = 0.437605
Epoch 206
Validation prc-auc = 0.409686
Epoch 207
Validation prc-auc = 0.405455
Epoch 208
Validation prc-auc = 0.400076
Epoch 209
Loss = 8.2000e-02, PNorm = 96.3277, GNorm = 2.6177, lr_0 = 7.0752e-04
Validation prc-auc = 0.392355
Epoch 210
Validation prc-auc = 0.392181
Epoch 211
Validation prc-auc = 0.395899
Epoch 212
Validation prc-auc = 0.400564
Epoch 213
Validation prc-auc = 0.374164
Epoch 214
Loss = 9.3076e-02, PNorm = 96.5050, GNorm = 3.4472, lr_0 = 7.0147e-04
Validation prc-auc = 0.391411
Epoch 215
Validation prc-auc = 0.440034
Epoch 216
Validation prc-auc = 0.431823
Epoch 217
Validation prc-auc = 0.443310
Epoch 218
Validation prc-auc = 0.444539
Epoch 219
Loss = 7.2743e-03, PNorm = 96.7375, GNorm = 0.2281, lr_0 = 6.9548e-04
Validation prc-auc = 0.439165
Epoch 220
Validation prc-auc = 0.439891
Epoch 221
Validation prc-auc = 0.425337
Epoch 222
Validation prc-auc = 0.421650
Epoch 223
Validation prc-auc = 0.420897
Epoch 224
Loss = 4.6742e-02, PNorm = 96.9103, GNorm = 0.3737, lr_0 = 6.8953e-04
Validation prc-auc = 0.404890
Epoch 225
Validation prc-auc = 0.397215
Epoch 226
Validation prc-auc = 0.390590
Epoch 227
Validation prc-auc = 0.390385
Epoch 228
Validation prc-auc = 0.398236
Epoch 229
Loss = 2.0346e-02, PNorm = 97.0351, GNorm = 0.5495, lr_0 = 6.8364e-04
Validation prc-auc = 0.435608
Epoch 230
Validation prc-auc = 0.436776
Epoch 231
Validation prc-auc = 0.438600
Epoch 232
Validation prc-auc = 0.420375
Epoch 233
Validation prc-auc = 0.426048
Epoch 234
Loss = 5.0229e-03, PNorm = 97.2166, GNorm = 0.1494, lr_0 = 6.7779e-04
Validation prc-auc = 0.433221
Epoch 235
Validation prc-auc = 0.434238
Epoch 236
Validation prc-auc = 0.439736
Epoch 237
Validation prc-auc = 0.443105
Epoch 238
Validation prc-auc = 0.428586
Epoch 239
Loss = 6.9298e-02, PNorm = 97.4045, GNorm = 0.8906, lr_0 = 6.7200e-04
Validation prc-auc = 0.432765
Epoch 240
Validation prc-auc = 0.425032
Epoch 241
Validation prc-auc = 0.439850
Epoch 242
Validation prc-auc = 0.448646
Epoch 243
Validation prc-auc = 0.444699
Epoch 244
Loss = 4.2373e-02, PNorm = 97.6631, GNorm = 1.4016, lr_0 = 6.6625e-04
Validation prc-auc = 0.431442
Epoch 245
Validation prc-auc = 0.437164
Epoch 246
Validation prc-auc = 0.440588
Epoch 247
Validation prc-auc = 0.438211
Epoch 248
Validation prc-auc = 0.437080
Epoch 249
Loss = 3.7743e-02, PNorm = 97.8981, GNorm = 1.4991, lr_0 = 6.6056e-04
Validation prc-auc = 0.436328
Epoch 250
Validation prc-auc = 0.422288
Epoch 251
Validation prc-auc = 0.421786
Epoch 252
Validation prc-auc = 0.432652
Epoch 253
Validation prc-auc = 0.431149
Epoch 254
Loss = 6.0635e-03, PNorm = 98.0762, GNorm = 0.2961, lr_0 = 6.5491e-04
Validation prc-auc = 0.422211
Epoch 255
Validation prc-auc = 0.418057
Epoch 256
Validation prc-auc = 0.427906
Epoch 257
Validation prc-auc = 0.420045
Epoch 258
Validation prc-auc = 0.420685
Epoch 259
Loss = 1.3345e-02, PNorm = 98.1923, GNorm = 0.4241, lr_0 = 6.4931e-04
Validation prc-auc = 0.420262
Epoch 260
Validation prc-auc = 0.419814
Epoch 261
Validation prc-auc = 0.413764
Epoch 262
Validation prc-auc = 0.422436
Epoch 263
Validation prc-auc = 0.413935
Epoch 264
Loss = 4.7539e-03, PNorm = 98.2805, GNorm = 0.0619, lr_0 = 6.4376e-04
Validation prc-auc = 0.411215
Epoch 265
Validation prc-auc = 0.408350
Epoch 266
Validation prc-auc = 0.388293
Epoch 267
Validation prc-auc = 0.390818
Epoch 268
Validation prc-auc = 0.391106
Epoch 269
Loss = 4.5988e-03, PNorm = 98.3555, GNorm = 0.0776, lr_0 = 6.3826e-04
Validation prc-auc = 0.391660
Epoch 270
Validation prc-auc = 0.391761
Epoch 271
Validation prc-auc = 0.392261
Epoch 272
Validation prc-auc = 0.392401
Epoch 273
Validation prc-auc = 0.393421
Epoch 274
Loss = 1.4887e-02, PNorm = 98.4088, GNorm = 0.3835, lr_0 = 6.3280e-04
Validation prc-auc = 0.395302
Epoch 275
Validation prc-auc = 0.397251
Epoch 276
Validation prc-auc = 0.396332
Epoch 277
Validation prc-auc = 0.401523
Epoch 278
Validation prc-auc = 0.404073
Epoch 279
Loss = 5.1243e-03, PNorm = 98.4518, GNorm = 0.2960, lr_0 = 6.2739e-04
Validation prc-auc = 0.405289
Epoch 280
Validation prc-auc = 0.406295
Epoch 281
Validation prc-auc = 0.407468
Epoch 282
Validation prc-auc = 0.407233
Epoch 283
Validation prc-auc = 0.407134
Epoch 284
Loss = 5.1230e-03, PNorm = 98.4836, GNorm = 0.4189, lr_0 = 6.2203e-04
Validation prc-auc = 0.404287
Epoch 285
Validation prc-auc = 0.401075
Epoch 286
Validation prc-auc = 0.401704
Epoch 287
Validation prc-auc = 0.402537
Epoch 288
Validation prc-auc = 0.404176
Epoch 289
Loss = 8.8714e-04, PNorm = 98.5025, GNorm = 0.0506, lr_0 = 6.1671e-04
Validation prc-auc = 0.404376
Epoch 290
Validation prc-auc = 0.404548
Epoch 291
Validation prc-auc = 0.403696
Epoch 292
Validation prc-auc = 0.403932
Epoch 293
Validation prc-auc = 0.396182
Epoch 294
Loss = 3.6529e-04, PNorm = 98.5148, GNorm = 0.0070, lr_0 = 6.1144e-04
Validation prc-auc = 0.397794
Epoch 295
Validation prc-auc = 0.399683
Epoch 296
Validation prc-auc = 0.401934
Epoch 297
Validation prc-auc = 0.402016
Epoch 298
Validation prc-auc = 0.402437
Epoch 299
Loss = 5.9027e-04, PNorm = 98.5220, GNorm = 0.0058, lr_0 = 6.0621e-04
Validation prc-auc = 0.402600
Model 0 best validation prc-auc = 0.470489 on epoch 156
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.200059
Ensemble test prc-auc = 0.200059
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.345296
Epoch 1
Validation prc-auc = 0.397953
Epoch 2
Validation prc-auc = 0.429859
Epoch 3
Validation prc-auc = 0.439972
Epoch 4
Loss = 5.4650e-01, PNorm = 87.5391, GNorm = 3.5155, lr_0 = 6.5000e-04
Validation prc-auc = 0.458672
Epoch 5
Validation prc-auc = 0.439217
Epoch 6
Validation prc-auc = 0.410210
Epoch 7
Validation prc-auc = 0.385668
Epoch 8
Validation prc-auc = 0.423867
Epoch 9
Loss = 4.8525e-01, PNorm = 87.8880, GNorm = 2.6083, lr_0 = 9.9743e-04
Validation prc-auc = 0.398844
Epoch 10
Validation prc-auc = 0.249633
Epoch 11
Validation prc-auc = 0.417868
Epoch 12
Validation prc-auc = 0.455599
Epoch 13
Validation prc-auc = 0.413701
Epoch 14
Loss = 5.5686e-01, PNorm = 88.2602, GNorm = 3.8282, lr_0 = 9.8890e-04
Validation prc-auc = 0.301431
Epoch 15
Validation prc-auc = 0.306078
Epoch 16
Validation prc-auc = 0.363840
Epoch 17
Validation prc-auc = 0.447965
Epoch 18
Validation prc-auc = 0.427806
Epoch 19
Loss = 4.2888e-01, PNorm = 88.5333, GNorm = 1.9630, lr_0 = 9.8045e-04
Validation prc-auc = 0.160067
Epoch 20
Validation prc-auc = 0.156999
Epoch 21
Validation prc-auc = 0.230521
Epoch 22
Validation prc-auc = 0.335513
Epoch 23
Validation prc-auc = 0.397613
Epoch 24
Loss = 2.9158e-01, PNorm = 88.7550, GNorm = 1.2853, lr_0 = 9.7207e-04
Validation prc-auc = 0.413415
Epoch 25
Validation prc-auc = 0.421106
Epoch 26
Validation prc-auc = 0.372082
Epoch 27
Validation prc-auc = 0.342842
Epoch 28
Validation prc-auc = 0.380344
Epoch 29
Loss = 2.9488e-01, PNorm = 88.9379, GNorm = 1.7945, lr_0 = 9.6376e-04
Validation prc-auc = 0.415807
Epoch 30
Validation prc-auc = 0.405616
Epoch 31
Validation prc-auc = 0.395222
Epoch 32
Validation prc-auc = 0.378046
Epoch 33
Validation prc-auc = 0.288881
Epoch 34
Loss = 2.5098e-01, PNorm = 89.0964, GNorm = 0.7755, lr_0 = 9.5552e-04
Validation prc-auc = 0.380208
Epoch 35
Validation prc-auc = 0.394693
Epoch 36
Validation prc-auc = 0.428095
Epoch 37
Validation prc-auc = 0.435981
Epoch 38
Validation prc-auc = 0.430127
Epoch 39
Loss = 2.8927e-01, PNorm = 89.2766, GNorm = 1.3117, lr_0 = 9.4735e-04
Validation prc-auc = 0.379995
Epoch 40
Validation prc-auc = 0.377980
Epoch 41
Validation prc-auc = 0.417914
Epoch 42
Validation prc-auc = 0.438948
Epoch 43
Validation prc-auc = 0.440644
Epoch 44
Loss = 2.5082e-01, PNorm = 89.4713, GNorm = 1.7440, lr_0 = 9.3925e-04
Validation prc-auc = 0.423018
Epoch 45
Validation prc-auc = 0.397087
Epoch 46
Validation prc-auc = 0.436960
Epoch 47
Validation prc-auc = 0.424882
Epoch 48
Validation prc-auc = 0.420032
Epoch 49
Loss = 1.7150e-01, PNorm = 89.6975, GNorm = 1.2054, lr_0 = 9.3122e-04
Validation prc-auc = 0.414458
Epoch 50
Validation prc-auc = 0.394320
Epoch 51
Validation prc-auc = 0.378413
Epoch 52
Validation prc-auc = 0.414205
Epoch 53
Validation prc-auc = 0.418748
Epoch 54
Loss = 1.4329e-01, PNorm = 89.9256, GNorm = 0.8593, lr_0 = 9.2326e-04
Validation prc-auc = 0.448786
Epoch 55
Validation prc-auc = 0.428325
Epoch 56
Validation prc-auc = 0.399453
Epoch 57
Validation prc-auc = 0.407004
Epoch 58
Validation prc-auc = 0.393900
Epoch 59
Loss = 1.4707e-01, PNorm = 90.1807, GNorm = 1.7485, lr_0 = 9.1537e-04
Validation prc-auc = 0.390683
Epoch 60
Validation prc-auc = 0.366941
Epoch 61
Validation prc-auc = 0.375070
Epoch 62
Validation prc-auc = 0.401751
Epoch 63
Validation prc-auc = 0.409856
Epoch 64
Loss = 1.5622e-01, PNorm = 90.4595, GNorm = 2.1492, lr_0 = 9.0754e-04
Validation prc-auc = 0.405847
Epoch 65
Validation prc-auc = 0.394544
Epoch 66
Validation prc-auc = 0.388452
Epoch 67
Validation prc-auc = 0.388061
Epoch 68
Validation prc-auc = 0.391796
Epoch 69
Loss = 1.3327e-01, PNorm = 90.7430, GNorm = 1.4150, lr_0 = 8.9978e-04
Validation prc-auc = 0.382405
Epoch 70
Validation prc-auc = 0.392910
Epoch 71
Validation prc-auc = 0.384088
Epoch 72
Validation prc-auc = 0.401304
Epoch 73
Validation prc-auc = 0.418237
Epoch 74
Loss = 9.1051e-02, PNorm = 91.0305, GNorm = 0.6922, lr_0 = 8.9209e-04
Validation prc-auc = 0.427026
Epoch 75
Validation prc-auc = 0.421611
Epoch 76
Validation prc-auc = 0.416828
Epoch 77
Validation prc-auc = 0.399165
Epoch 78
Validation prc-auc = 0.402081
Epoch 79
Loss = 6.0398e-02, PNorm = 91.3246, GNorm = 1.2415, lr_0 = 8.8447e-04
Validation prc-auc = 0.401793
Epoch 80
Validation prc-auc = 0.391863
Epoch 81
Validation prc-auc = 0.395779
Epoch 82
Validation prc-auc = 0.410588
Epoch 83
Validation prc-auc = 0.421463
Epoch 84
Loss = 3.2777e-02, PNorm = 91.5794, GNorm = 0.3374, lr_0 = 8.7691e-04
Validation prc-auc = 0.431160
Epoch 85
Validation prc-auc = 0.426399
Epoch 86
Validation prc-auc = 0.417326
Epoch 87
Validation prc-auc = 0.420560
Epoch 88
Validation prc-auc = 0.431729
Epoch 89
Loss = 1.0467e-01, PNorm = 91.8236, GNorm = 1.3608, lr_0 = 8.6941e-04
Validation prc-auc = 0.429769
Epoch 90
Validation prc-auc = 0.439312
Epoch 91
Validation prc-auc = 0.424395
Epoch 92
Validation prc-auc = 0.386673
Epoch 93
Validation prc-auc = 0.388580
Epoch 94
Loss = 4.1776e-02, PNorm = 92.1045, GNorm = 0.3250, lr_0 = 8.6198e-04
Validation prc-auc = 0.408739
Epoch 95
Validation prc-auc = 0.390275
Epoch 96
Validation prc-auc = 0.399803
Epoch 97
Validation prc-auc = 0.395132
Epoch 98
Validation prc-auc = 0.416611
Epoch 99
Loss = 5.5576e-02, PNorm = 92.4234, GNorm = 0.8673, lr_0 = 8.5461e-04
Validation prc-auc = 0.426399
Epoch 100
Validation prc-auc = 0.432530
Epoch 101
Validation prc-auc = 0.411476
Epoch 102
Validation prc-auc = 0.418449
Epoch 103
Validation prc-auc = 0.417141
Epoch 104
Loss = 1.0132e-01, PNorm = 92.7123, GNorm = 1.8984, lr_0 = 8.4730e-04
Validation prc-auc = 0.402824
Epoch 105
Validation prc-auc = 0.390813
Epoch 106
Validation prc-auc = 0.403827
Epoch 107
Validation prc-auc = 0.436462
Epoch 108
Validation prc-auc = 0.453781
Epoch 109
Loss = 6.7348e-02, PNorm = 93.0376, GNorm = 1.8721, lr_0 = 8.4006e-04
Validation prc-auc = 0.447966
Epoch 110
Validation prc-auc = 0.419682
Epoch 111
Validation prc-auc = 0.365763
Epoch 112
Validation prc-auc = 0.385110
Epoch 113
Validation prc-auc = 0.398783
Epoch 114
Loss = 1.8176e-02, PNorm = 93.3244, GNorm = 0.4433, lr_0 = 8.3288e-04
Validation prc-auc = 0.413442
Epoch 115
Validation prc-auc = 0.410537
Epoch 116
Validation prc-auc = 0.387063
Epoch 117
Validation prc-auc = 0.396412
Epoch 118
Validation prc-auc = 0.376231
Epoch 119
Loss = 1.3403e-02, PNorm = 93.5562, GNorm = 0.2899, lr_0 = 8.2576e-04
Validation prc-auc = 0.280810
Epoch 120
Validation prc-auc = 0.359464
Epoch 121
Validation prc-auc = 0.424847
Epoch 122
Validation prc-auc = 0.403089
Epoch 123
Validation prc-auc = 0.430154
Epoch 124
Loss = 9.0079e-02, PNorm = 93.7889, GNorm = 1.2111, lr_0 = 8.1870e-04
Validation prc-auc = 0.377264
Epoch 125
Validation prc-auc = 0.376365
Epoch 126
Validation prc-auc = 0.415004
Epoch 127
Validation prc-auc = 0.418642
Epoch 128
Validation prc-auc = 0.431878
Epoch 129
Loss = 9.4177e-02, PNorm = 94.0358, GNorm = 1.1968, lr_0 = 8.1170e-04
Validation prc-auc = 0.424261
Epoch 130
Validation prc-auc = 0.421287
Epoch 131
Validation prc-auc = 0.454643
Epoch 132
Validation prc-auc = 0.462618
Epoch 133
Validation prc-auc = 0.451899
Epoch 134
Loss = 2.2843e-02, PNorm = 94.2828, GNorm = 0.5677, lr_0 = 8.0476e-04
Validation prc-auc = 0.444299
Epoch 135
Validation prc-auc = 0.458585
Epoch 136
Validation prc-auc = 0.447595
Epoch 137
Validation prc-auc = 0.434736
Epoch 138
Validation prc-auc = 0.435275
Epoch 139
Loss = 5.9239e-02, PNorm = 94.5041, GNorm = 0.6286, lr_0 = 7.9788e-04
Validation prc-auc = 0.446905
Epoch 140
Validation prc-auc = 0.425588
Epoch 141
Validation prc-auc = 0.381641
Epoch 142
Validation prc-auc = 0.409263
Epoch 143
Validation prc-auc = 0.451313
Epoch 144
Loss = 1.8558e-02, PNorm = 94.6963, GNorm = 0.5509, lr_0 = 7.9106e-04
Validation prc-auc = 0.438574
Epoch 145
Validation prc-auc = 0.419515
Epoch 146
Validation prc-auc = 0.379284
Epoch 147
Validation prc-auc = 0.357701
Epoch 148
Validation prc-auc = 0.397665
Epoch 149
Loss = 6.9395e-03, PNorm = 94.9323, GNorm = 0.2846, lr_0 = 7.8430e-04
Validation prc-auc = 0.420249
Epoch 150
Validation prc-auc = 0.421711
Epoch 151
Validation prc-auc = 0.443405
Epoch 152
Validation prc-auc = 0.439824
Epoch 153
Validation prc-auc = 0.448457
Epoch 154
Loss = 3.9413e-02, PNorm = 95.2034, GNorm = 1.1873, lr_0 = 7.7759e-04
Validation prc-auc = 0.431966
Epoch 155
Validation prc-auc = 0.431441
Epoch 156
Validation prc-auc = 0.432979
Epoch 157
Validation prc-auc = 0.431180
Epoch 158
Validation prc-auc = 0.442532
Epoch 159
Loss = 2.3090e-02, PNorm = 95.4389, GNorm = 0.8423, lr_0 = 7.7095e-04
Validation prc-auc = 0.427278
Epoch 160
Validation prc-auc = 0.443910
Epoch 161
Validation prc-auc = 0.442240
Epoch 162
Validation prc-auc = 0.440824
Epoch 163
Validation prc-auc = 0.439454
Epoch 164
Loss = 4.4303e-02, PNorm = 95.6515, GNorm = 0.9401, lr_0 = 7.6436e-04
Validation prc-auc = 0.452679
Epoch 165
Validation prc-auc = 0.457580
Epoch 166
Validation prc-auc = 0.444867
Epoch 167
Validation prc-auc = 0.454049
Epoch 168
Validation prc-auc = 0.456079
Epoch 169
Loss = 9.7944e-02, PNorm = 95.8651, GNorm = 2.7679, lr_0 = 7.5782e-04
Validation prc-auc = 0.438044
Epoch 170
Validation prc-auc = 0.430740
Epoch 171
Validation prc-auc = 0.429836
Epoch 172
Validation prc-auc = 0.413219
Epoch 173
Validation prc-auc = 0.316124
Epoch 174
Loss = 2.8500e-02, PNorm = 96.0496, GNorm = 0.8279, lr_0 = 7.5134e-04
Validation prc-auc = 0.364934
Epoch 175
Validation prc-auc = 0.412449
Epoch 176
Validation prc-auc = 0.435304
Epoch 177
Validation prc-auc = 0.435455
Epoch 178
Validation prc-auc = 0.441816
Epoch 179
Loss = 7.4814e-02, PNorm = 96.2177, GNorm = 0.4847, lr_0 = 7.4492e-04
Validation prc-auc = 0.439187
Epoch 180
Validation prc-auc = 0.438297
Epoch 181
Validation prc-auc = 0.438177
Epoch 182
Validation prc-auc = 0.437811
Epoch 183
Validation prc-auc = 0.437770
Epoch 184
Loss = 1.5604e-02, PNorm = 96.3513, GNorm = 0.4570, lr_0 = 7.3855e-04
Validation prc-auc = 0.438316
Epoch 185
Validation prc-auc = 0.439326
Epoch 186
Validation prc-auc = 0.438247
Epoch 187
Validation prc-auc = 0.437819
Epoch 188
Validation prc-auc = 0.438239
Epoch 189
Loss = 7.8601e-03, PNorm = 96.4434, GNorm = 0.3414, lr_0 = 7.3224e-04
Validation prc-auc = 0.437647
Epoch 190
Validation prc-auc = 0.437335
Epoch 191
Validation prc-auc = 0.436330
Epoch 192
Validation prc-auc = 0.435776
Epoch 193
Validation prc-auc = 0.433379
Epoch 194
Loss = 1.7372e-02, PNorm = 96.5181, GNorm = 0.9535, lr_0 = 7.2598e-04
Validation prc-auc = 0.416018
Epoch 195
Validation prc-auc = 0.409377
Epoch 196
Validation prc-auc = 0.411884
Epoch 197
Validation prc-auc = 0.422987
Epoch 198
Validation prc-auc = 0.432197
Epoch 199
Loss = 2.6361e-03, PNorm = 96.6258, GNorm = 0.0999, lr_0 = 7.1977e-04
Validation prc-auc = 0.433328
Epoch 200
Validation prc-auc = 0.436127
Epoch 201
Validation prc-auc = 0.439036
Epoch 202
Validation prc-auc = 0.439385
Epoch 203
Validation prc-auc = 0.439423
Epoch 204
Loss = 3.7112e-02, PNorm = 96.7030, GNorm = 1.6086, lr_0 = 7.1362e-04
Validation prc-auc = 0.437479
Epoch 205
Validation prc-auc = 0.434739
Epoch 206
Validation prc-auc = 0.451632
Epoch 207
Validation prc-auc = 0.412239
Epoch 208
Validation prc-auc = 0.420861
Epoch 209
Loss = 4.6180e-02, PNorm = 96.8165, GNorm = 1.2163, lr_0 = 7.0752e-04
Validation prc-auc = 0.423229
Epoch 210
Validation prc-auc = 0.431348
Epoch 211
Validation prc-auc = 0.427959
Epoch 212
Validation prc-auc = 0.421449
Epoch 213
Validation prc-auc = 0.418301
Epoch 214
Loss = 1.0135e-02, PNorm = 97.0482, GNorm = 0.4283, lr_0 = 7.0147e-04
Validation prc-auc = 0.420951
Epoch 215
Validation prc-auc = 0.419704
Epoch 216
Validation prc-auc = 0.416131
Epoch 217
Validation prc-auc = 0.410020
Epoch 218
Validation prc-auc = 0.426470
Epoch 219
Loss = 1.8448e-02, PNorm = 97.2398, GNorm = 0.9302, lr_0 = 6.9548e-04
Validation prc-auc = 0.434016
Epoch 220
Validation prc-auc = 0.443671
Epoch 221
Validation prc-auc = 0.445657
Epoch 222
Validation prc-auc = 0.448002
Epoch 223
Validation prc-auc = 0.449036
Epoch 224
Loss = 5.5950e-03, PNorm = 97.3631, GNorm = 0.1118, lr_0 = 6.8953e-04
Validation prc-auc = 0.452219
Epoch 225
Validation prc-auc = 0.452943
Epoch 226
Validation prc-auc = 0.452643
Epoch 227
Validation prc-auc = 0.453714
Epoch 228
Validation prc-auc = 0.449453
Epoch 229
Loss = 2.7372e-03, PNorm = 97.4618, GNorm = 0.0243, lr_0 = 6.8364e-04
Validation prc-auc = 0.448900
Epoch 230
Validation prc-auc = 0.447521
Epoch 231
Validation prc-auc = 0.446667
Epoch 232
Validation prc-auc = 0.446778
Epoch 233
Validation prc-auc = 0.446791
Epoch 234
Loss = 3.1351e-03, PNorm = 97.5393, GNorm = 0.1289, lr_0 = 6.7779e-04
Validation prc-auc = 0.448656
Epoch 235
Validation prc-auc = 0.448095
Epoch 236
Validation prc-auc = 0.447374
Epoch 237
Validation prc-auc = 0.450302
Epoch 238
Validation prc-auc = 0.426189
Epoch 239
Loss = 1.6842e-03, PNorm = 97.6130, GNorm = 0.0885, lr_0 = 6.7200e-04
Validation prc-auc = 0.426695
Epoch 240
Validation prc-auc = 0.419112
Epoch 241
Validation prc-auc = 0.417997
Epoch 242
Validation prc-auc = 0.417896
Epoch 243
Validation prc-auc = 0.412258
Epoch 244
Loss = 4.2356e-02, PNorm = 97.6621, GNorm = 0.0270, lr_0 = 6.6625e-04
Validation prc-auc = 0.440429
Epoch 245
Validation prc-auc = 0.439198
Epoch 246
Validation prc-auc = 0.439943
Epoch 247
Validation prc-auc = 0.441446
Epoch 248
Validation prc-auc = 0.439731
Epoch 249
Loss = 3.2602e-03, PNorm = 97.7135, GNorm = 0.1280, lr_0 = 6.6056e-04
Validation prc-auc = 0.453987
Epoch 250
Validation prc-auc = 0.436301
Epoch 251
Validation prc-auc = 0.435568
Epoch 252
Validation prc-auc = 0.434950
Epoch 253
Validation prc-auc = 0.433875
Epoch 254
Loss = 8.3003e-04, PNorm = 97.7511, GNorm = 0.0377, lr_0 = 6.5491e-04
Validation prc-auc = 0.433869
Epoch 255
Validation prc-auc = 0.434107
Epoch 256
Validation prc-auc = 0.434384
Epoch 257
Validation prc-auc = 0.433799
Epoch 258
Validation prc-auc = 0.433462
Epoch 259
Loss = 5.3557e-04, PNorm = 97.7770, GNorm = 0.0334, lr_0 = 6.4931e-04
Validation prc-auc = 0.414508
Epoch 260
Validation prc-auc = 0.414099
Epoch 261
Validation prc-auc = 0.413484
Epoch 262
Validation prc-auc = 0.407508
Epoch 263
Validation prc-auc = 0.406447
Epoch 264
Loss = 1.0900e-03, PNorm = 97.8071, GNorm = 0.0321, lr_0 = 6.4376e-04
Validation prc-auc = 0.405769
Epoch 265
Validation prc-auc = 0.405217
Epoch 266
Validation prc-auc = 0.430413
Epoch 267
Validation prc-auc = 0.430056
Epoch 268
Validation prc-auc = 0.429296
Epoch 269
Loss = 2.4063e-04, PNorm = 97.8425, GNorm = 0.0042, lr_0 = 6.3826e-04
Validation prc-auc = 0.428294
Epoch 270
Validation prc-auc = 0.428163
Epoch 271
Validation prc-auc = 0.427744
Epoch 272
Validation prc-auc = 0.427342
Epoch 273
Validation prc-auc = 0.427548
Epoch 274
Loss = 4.1288e-03, PNorm = 97.8635, GNorm = 0.3955, lr_0 = 6.3280e-04
Validation prc-auc = 0.427765
Epoch 275
Validation prc-auc = 0.427608
Epoch 276
Validation prc-auc = 0.427682
Epoch 277
Validation prc-auc = 0.427935
Epoch 278
Validation prc-auc = 0.428191
Epoch 279
Loss = 2.3262e-04, PNorm = 97.8806, GNorm = 0.0086, lr_0 = 6.2739e-04
Validation prc-auc = 0.427988
Epoch 280
Validation prc-auc = 0.429788
Epoch 281
Validation prc-auc = 0.417998
Epoch 282
Validation prc-auc = 0.399640
Epoch 283
Validation prc-auc = 0.400489
Epoch 284
Loss = 6.9859e-04, PNorm = 97.9103, GNorm = 0.0244, lr_0 = 6.2203e-04
Validation prc-auc = 0.396902
Epoch 285
Validation prc-auc = 0.397008
Epoch 286
Validation prc-auc = 0.391353
Epoch 287
Validation prc-auc = 0.390284
Epoch 288
Validation prc-auc = 0.397830
Epoch 289
Loss = 2.3861e-02, PNorm = 97.9395, GNorm = 1.7585, lr_0 = 6.1671e-04
Validation prc-auc = 0.406299
Epoch 290
Validation prc-auc = 0.396441
Epoch 291
Validation prc-auc = 0.393739
Epoch 292
Validation prc-auc = 0.363331
Epoch 293
Validation prc-auc = 0.362041
Epoch 294
Loss = 4.5186e-02, PNorm = 98.0088, GNorm = 0.0242, lr_0 = 6.1144e-04
Validation prc-auc = 0.396758
Epoch 295
Validation prc-auc = 0.404114
Epoch 296
Validation prc-auc = 0.412315
Epoch 297
Validation prc-auc = 0.413483
Epoch 298
Validation prc-auc = 0.412507
Epoch 299
Loss = 1.4485e-02, PNorm = 98.0701, GNorm = 0.8399, lr_0 = 6.0621e-04
Validation prc-auc = 0.412754
Model 0 best validation prc-auc = 0.462618 on epoch 132
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.261625
Ensemble test prc-auc = 0.261625
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.347106
Epoch 1
Validation prc-auc = 0.366081
Epoch 2
Validation prc-auc = 0.314161
Epoch 3
Validation prc-auc = 0.141773
Epoch 4
Loss = 6.4507e-01, PNorm = 87.5310, GNorm = 2.6047, lr_0 = 6.5000e-04
Validation prc-auc = 0.272102
Epoch 5
Validation prc-auc = 0.415886
Epoch 6
Validation prc-auc = 0.399809
Epoch 7
Validation prc-auc = 0.425047
Epoch 8
Validation prc-auc = 0.186940
Epoch 9
Loss = 4.8665e-01, PNorm = 87.8775, GNorm = 1.4454, lr_0 = 9.9743e-04
Validation prc-auc = 0.168378
Epoch 10
Validation prc-auc = 0.344957
Epoch 11
Validation prc-auc = 0.429849
Epoch 12
Validation prc-auc = 0.406577
Epoch 13
Validation prc-auc = 0.309375
Epoch 14
Loss = 3.3776e-01, PNorm = 88.2496, GNorm = 1.1852, lr_0 = 9.8890e-04
Validation prc-auc = 0.312110
Epoch 15
Validation prc-auc = 0.335441
Epoch 16
Validation prc-auc = 0.387587
Epoch 17
Validation prc-auc = 0.426543
Epoch 18
Validation prc-auc = 0.335044
Epoch 19
Loss = 3.1148e-01, PNorm = 88.5137, GNorm = 1.2991, lr_0 = 9.8045e-04
Validation prc-auc = 0.191357
Epoch 20
Validation prc-auc = 0.341650
Epoch 21
Validation prc-auc = 0.438590
Epoch 22
Validation prc-auc = 0.379619
Epoch 23
Validation prc-auc = 0.323692
Epoch 24
Loss = 2.5704e-01, PNorm = 88.7235, GNorm = 0.5544, lr_0 = 9.7207e-04
Validation prc-auc = 0.416202
Epoch 25
Validation prc-auc = 0.395876
Epoch 26
Validation prc-auc = 0.371324
Epoch 27
Validation prc-auc = 0.312787
Epoch 28
Validation prc-auc = 0.352437
Epoch 29
Loss = 3.1346e-01, PNorm = 88.9030, GNorm = 1.5812, lr_0 = 9.6376e-04
Validation prc-auc = 0.385020
Epoch 30
Validation prc-auc = 0.380186
Epoch 31
Validation prc-auc = 0.332111
Epoch 32
Validation prc-auc = 0.309958
Epoch 33
Validation prc-auc = 0.333999
Epoch 34
Loss = 3.1598e-01, PNorm = 89.0821, GNorm = 0.8170, lr_0 = 9.5552e-04
Validation prc-auc = 0.290940
Epoch 35
Validation prc-auc = 0.270662
Epoch 36
Validation prc-auc = 0.357380
Epoch 37
Validation prc-auc = 0.441355
Epoch 38
Validation prc-auc = 0.447320
Epoch 39
Loss = 1.8527e-01, PNorm = 89.2615, GNorm = 1.6925, lr_0 = 9.4735e-04
Validation prc-auc = 0.410295
Epoch 40
Validation prc-auc = 0.375697
Epoch 41
Validation prc-auc = 0.379144
Epoch 42
Validation prc-auc = 0.383371
Epoch 43
Validation prc-auc = 0.398204
Epoch 44
Loss = 2.1268e-01, PNorm = 89.4535, GNorm = 1.8270, lr_0 = 9.3925e-04
Validation prc-auc = 0.379413
Epoch 45
Validation prc-auc = 0.410074
Epoch 46
Validation prc-auc = 0.421839
Epoch 47
Validation prc-auc = 0.419215
Epoch 48
Validation prc-auc = 0.408009
Epoch 49
Loss = 1.3781e-01, PNorm = 89.6588, GNorm = 1.3967, lr_0 = 9.3122e-04
Validation prc-auc = 0.355542
Epoch 50
Validation prc-auc = 0.430225
Epoch 51
Validation prc-auc = 0.440110
Epoch 52
Validation prc-auc = 0.433767
Epoch 53
Validation prc-auc = 0.305457
Epoch 54
Loss = 2.5570e-01, PNorm = 89.8739, GNorm = 2.8644, lr_0 = 9.2326e-04
Validation prc-auc = 0.272992
Epoch 55
Validation prc-auc = 0.462601
Epoch 56
Validation prc-auc = 0.453591
Epoch 57
Validation prc-auc = 0.453710
Epoch 58
Validation prc-auc = 0.426763
Epoch 59
Loss = 2.8312e-01, PNorm = 90.1417, GNorm = 1.0763, lr_0 = 9.1537e-04
Validation prc-auc = 0.269933
Epoch 60
Validation prc-auc = 0.213577
Epoch 61
Validation prc-auc = 0.222806
Epoch 62
Validation prc-auc = 0.267732
Epoch 63
Validation prc-auc = 0.401287
Epoch 64
Loss = 2.0369e-01, PNorm = 90.3977, GNorm = 2.2718, lr_0 = 9.0754e-04
Validation prc-auc = 0.446672
Epoch 65
Validation prc-auc = 0.459244
Epoch 66
Validation prc-auc = 0.449279
Epoch 67
Validation prc-auc = 0.455307
Epoch 68
Validation prc-auc = 0.407901
Epoch 69
Loss = 1.0982e-01, PNorm = 90.6414, GNorm = 1.0647, lr_0 = 8.9978e-04
Validation prc-auc = 0.388086
Epoch 70
Validation prc-auc = 0.425145
Epoch 71
Validation prc-auc = 0.465142
Epoch 72
Validation prc-auc = 0.460101
Epoch 73
Validation prc-auc = 0.466590
Epoch 74
Loss = 1.3156e-01, PNorm = 90.8741, GNorm = 1.3887, lr_0 = 8.9209e-04
Validation prc-auc = 0.462199
Epoch 75
Validation prc-auc = 0.446432
Epoch 76
Validation prc-auc = 0.412782
Epoch 77
Validation prc-auc = 0.410136
Epoch 78
Validation prc-auc = 0.417341
Epoch 79
Loss = 8.4901e-02, PNorm = 91.1304, GNorm = 1.0014, lr_0 = 8.8447e-04
Validation prc-auc = 0.436940
Epoch 80
Validation prc-auc = 0.421253
Epoch 81
Validation prc-auc = 0.410094
Epoch 82
Validation prc-auc = 0.419629
Epoch 83
Validation prc-auc = 0.413835
Epoch 84
Loss = 1.4735e-01, PNorm = 91.4107, GNorm = 1.1256, lr_0 = 8.7691e-04
Validation prc-auc = 0.418869
Epoch 85
Validation prc-auc = 0.412618
Epoch 86
Validation prc-auc = 0.428706
Epoch 87
Validation prc-auc = 0.422926
Epoch 88
Validation prc-auc = 0.409245
Epoch 89
Loss = 7.5222e-02, PNorm = 91.6716, GNorm = 0.7904, lr_0 = 8.6941e-04
Validation prc-auc = 0.391422
Epoch 90
Validation prc-auc = 0.401364
Epoch 91
Validation prc-auc = 0.421808
Epoch 92
Validation prc-auc = 0.428935
Epoch 93
Validation prc-auc = 0.427654
Epoch 94
Loss = 5.1916e-02, PNorm = 91.9141, GNorm = 1.3283, lr_0 = 8.6198e-04
Validation prc-auc = 0.410436
Epoch 95
Validation prc-auc = 0.432512
Epoch 96
Validation prc-auc = 0.439406
Epoch 97
Validation prc-auc = 0.439636
Epoch 98
Validation prc-auc = 0.425271
Epoch 99
Loss = 1.7727e-01, PNorm = 92.1641, GNorm = 2.6703, lr_0 = 8.5461e-04
Validation prc-auc = 0.425175
Epoch 100
Validation prc-auc = 0.415358
Epoch 101
Validation prc-auc = 0.427934
Epoch 102
Validation prc-auc = 0.424493
Epoch 103
Validation prc-auc = 0.420560
Epoch 104
Loss = 4.5942e-02, PNorm = 92.4474, GNorm = 0.8205, lr_0 = 8.4730e-04
Validation prc-auc = 0.433176
Epoch 105
Validation prc-auc = 0.438260
Epoch 106
Validation prc-auc = 0.445269
Epoch 107
Validation prc-auc = 0.454161
Epoch 108
Validation prc-auc = 0.447567
Epoch 109
Loss = 9.8193e-02, PNorm = 92.6848, GNorm = 1.9164, lr_0 = 8.4006e-04
Validation prc-auc = 0.426684
Epoch 110
Validation prc-auc = 0.426308
Epoch 111
Validation prc-auc = 0.420972
Epoch 112
Validation prc-auc = 0.430302
Epoch 113
Validation prc-auc = 0.450282
Epoch 114
Loss = 7.1543e-02, PNorm = 92.9264, GNorm = 1.3952, lr_0 = 8.3288e-04
Validation prc-auc = 0.457898
Epoch 115
Validation prc-auc = 0.456335
Epoch 116
Validation prc-auc = 0.467182
Epoch 117
Validation prc-auc = 0.447039
Epoch 118
Validation prc-auc = 0.438782
Epoch 119
Loss = 3.5328e-02, PNorm = 93.1249, GNorm = 0.5076, lr_0 = 8.2576e-04
Validation prc-auc = 0.450934
Epoch 120
Validation prc-auc = 0.458655
Epoch 121
Validation prc-auc = 0.455194
Epoch 122
Validation prc-auc = 0.470009
Epoch 123
Validation prc-auc = 0.455325
Epoch 124
Loss = 4.0500e-02, PNorm = 93.2971, GNorm = 0.9234, lr_0 = 8.1870e-04
Validation prc-auc = 0.450480
Epoch 125
Validation prc-auc = 0.416326
Epoch 126
Validation prc-auc = 0.401599
Epoch 127
Validation prc-auc = 0.374380
Epoch 128
Validation prc-auc = 0.341822
Epoch 129
Loss = 1.2726e-01, PNorm = 93.5529, GNorm = 0.7897, lr_0 = 8.1170e-04
Validation prc-auc = 0.355322
Epoch 130
Validation prc-auc = 0.345619
Epoch 131
Validation prc-auc = 0.444769
Epoch 132
Validation prc-auc = 0.471051
Epoch 133
Validation prc-auc = 0.473397
Epoch 134
Loss = 5.2099e-02, PNorm = 93.8087, GNorm = 0.6003, lr_0 = 8.0476e-04
Validation prc-auc = 0.458734
Epoch 135
Validation prc-auc = 0.401211
Epoch 136
Validation prc-auc = 0.406514
Epoch 137
Validation prc-auc = 0.433566
Epoch 138
Validation prc-auc = 0.454056
Epoch 139
Loss = 5.9880e-02, PNorm = 94.0601, GNorm = 0.5249, lr_0 = 7.9788e-04
Validation prc-auc = 0.448589
Epoch 140
Validation prc-auc = 0.419378
Epoch 141
Validation prc-auc = 0.417053
Epoch 142
Validation prc-auc = 0.449423
Epoch 143
Validation prc-auc = 0.461785
Epoch 144
Loss = 5.3564e-02, PNorm = 94.2845, GNorm = 0.7284, lr_0 = 7.9106e-04
Validation prc-auc = 0.451613
Epoch 145
Validation prc-auc = 0.476246
Epoch 146
Validation prc-auc = 0.464441
Epoch 147
Validation prc-auc = 0.454768
Epoch 148
Validation prc-auc = 0.438643
Epoch 149
Loss = 7.5154e-02, PNorm = 94.5175, GNorm = 1.9858, lr_0 = 7.8430e-04
Validation prc-auc = 0.409285
Epoch 150
Validation prc-auc = 0.385199
Epoch 151
Validation prc-auc = 0.390479
Epoch 152
Validation prc-auc = 0.393683
Epoch 153
Validation prc-auc = 0.441892
Epoch 154
Loss = 3.3971e-02, PNorm = 94.7322, GNorm = 0.8888, lr_0 = 7.7759e-04
Validation prc-auc = 0.462207
Epoch 155
Validation prc-auc = 0.463593
Epoch 156
Validation prc-auc = 0.454706
Epoch 157
Validation prc-auc = 0.452430
Epoch 158
Validation prc-auc = 0.466755
Epoch 159
Loss = 2.1518e-02, PNorm = 94.9514, GNorm = 0.7119, lr_0 = 7.7095e-04
Validation prc-auc = 0.465851
Epoch 160
Validation prc-auc = 0.463494
Epoch 161
Validation prc-auc = 0.450962
Epoch 162
Validation prc-auc = 0.440767
Epoch 163
Validation prc-auc = 0.434797
Epoch 164
Loss = 2.0108e-02, PNorm = 95.1552, GNorm = 0.4747, lr_0 = 7.6436e-04
Validation prc-auc = 0.445498
Epoch 165
Validation prc-auc = 0.442026
Epoch 166
Validation prc-auc = 0.445929
Epoch 167
Validation prc-auc = 0.460364
Epoch 168
Validation prc-auc = 0.443677
Epoch 169
Loss = 4.7617e-02, PNorm = 95.3147, GNorm = 0.1372, lr_0 = 7.5782e-04
Validation prc-auc = 0.441999
Epoch 170
Validation prc-auc = 0.446606
Epoch 171
Validation prc-auc = 0.431010
Epoch 172
Validation prc-auc = 0.413602
Epoch 173
Validation prc-auc = 0.410075
Epoch 174
Loss = 2.5343e-02, PNorm = 95.4489, GNorm = 0.0236, lr_0 = 7.5134e-04
Validation prc-auc = 0.430495
Epoch 175
Validation prc-auc = 0.449386
Epoch 176
Validation prc-auc = 0.436549
Epoch 177
Validation prc-auc = 0.434583
Epoch 178
Validation prc-auc = 0.417644
Epoch 179
Loss = 1.4815e-02, PNorm = 95.5503, GNorm = 0.6524, lr_0 = 7.4492e-04
Validation prc-auc = 0.417634
Epoch 180
Validation prc-auc = 0.415905
Epoch 181
Validation prc-auc = 0.410078
Epoch 182
Validation prc-auc = 0.385834
Epoch 183
Validation prc-auc = 0.363401
Epoch 184
Loss = 3.2900e-02, PNorm = 95.6445, GNorm = 0.7433, lr_0 = 7.3855e-04
Validation prc-auc = 0.379663
Epoch 185
Validation prc-auc = 0.394129
Epoch 186
Validation prc-auc = 0.429384
Epoch 187
Validation prc-auc = 0.447012
Epoch 188
Validation prc-auc = 0.459601
Epoch 189
Loss = 1.6493e-02, PNorm = 95.7500, GNorm = 0.7462, lr_0 = 7.3224e-04
Validation prc-auc = 0.440779
Epoch 190
Validation prc-auc = 0.442781
Epoch 191
Validation prc-auc = 0.446059
Epoch 192
Validation prc-auc = 0.444621
Epoch 193
Validation prc-auc = 0.448014
Epoch 194
Loss = 1.2165e-03, PNorm = 95.8894, GNorm = 0.0783, lr_0 = 7.2598e-04
Validation prc-auc = 0.437061
Epoch 195
Validation prc-auc = 0.430464
Epoch 196
Validation prc-auc = 0.432675
Epoch 197
Validation prc-auc = 0.445684
Epoch 198
Validation prc-auc = 0.457407
Epoch 199
Loss = 2.2319e-03, PNorm = 95.9907, GNorm = 0.0519, lr_0 = 7.1977e-04
Validation prc-auc = 0.455573
Epoch 200
Validation prc-auc = 0.455727
Epoch 201
Validation prc-auc = 0.457288
Epoch 202
Validation prc-auc = 0.461536
Epoch 203
Validation prc-auc = 0.461723
Epoch 204
Loss = 2.9703e-02, PNorm = 96.0571, GNorm = 0.4934, lr_0 = 7.1362e-04
Validation prc-auc = 0.459681
Epoch 205
Validation prc-auc = 0.465384
Epoch 206
Validation prc-auc = 0.451531
Epoch 207
Validation prc-auc = 0.448034
Epoch 208
Validation prc-auc = 0.450118
Epoch 209
Loss = 7.4246e-02, PNorm = 96.1344, GNorm = 2.5891, lr_0 = 7.0752e-04
Validation prc-auc = 0.448799
Epoch 210
Validation prc-auc = 0.456485
Epoch 211
Validation prc-auc = 0.452803
Epoch 212
Validation prc-auc = 0.441251
Epoch 213
Validation prc-auc = 0.438024
Epoch 214
Loss = 1.5476e-02, PNorm = 96.2304, GNorm = 0.0608, lr_0 = 7.0147e-04
Validation prc-auc = 0.452988
Epoch 215
Validation prc-auc = 0.454233
Epoch 216
Validation prc-auc = 0.454205
Epoch 217
Validation prc-auc = 0.443615
Epoch 218
Validation prc-auc = 0.423541
Epoch 219
Loss = 1.5335e-02, PNorm = 96.3076, GNorm = 0.7071, lr_0 = 6.9548e-04
Validation prc-auc = 0.419971
Epoch 220
Validation prc-auc = 0.430386
Epoch 221
Validation prc-auc = 0.433054
Epoch 222
Validation prc-auc = 0.442086
Epoch 223
Validation prc-auc = 0.444818
Epoch 224
Loss = 6.8381e-02, PNorm = 96.3787, GNorm = 1.7497, lr_0 = 6.8953e-04
Validation prc-auc = 0.453413
Epoch 225
Validation prc-auc = 0.436242
Epoch 226
Validation prc-auc = 0.410956
Epoch 227
Validation prc-auc = 0.360851
Epoch 228
Validation prc-auc = 0.363926
Epoch 229
Loss = 1.2282e-02, PNorm = 96.5002, GNorm = 0.3328, lr_0 = 6.8364e-04
Validation prc-auc = 0.362967
Epoch 230
Validation prc-auc = 0.377880
Epoch 231
Validation prc-auc = 0.423705
Epoch 232
Validation prc-auc = 0.445170
Epoch 233
Validation prc-auc = 0.446895
Epoch 234
Loss = 1.2478e-02, PNorm = 96.6570, GNorm = 0.1290, lr_0 = 6.7779e-04
Validation prc-auc = 0.453398
Epoch 235
Validation prc-auc = 0.452336
Epoch 236
Validation prc-auc = 0.424656
Epoch 237
Validation prc-auc = 0.415037
Epoch 238
Validation prc-auc = 0.411177
Epoch 239
Loss = 3.5422e-03, PNorm = 96.8139, GNorm = 0.2591, lr_0 = 6.7200e-04
Validation prc-auc = 0.405807
Epoch 240
Validation prc-auc = 0.404466
Epoch 241
Validation prc-auc = 0.406081
Epoch 242
Validation prc-auc = 0.406357
Epoch 243
Validation prc-auc = 0.427698
Epoch 244
Loss = 5.9168e-02, PNorm = 97.0097, GNorm = 1.8952, lr_0 = 6.6625e-04
Validation prc-auc = 0.420155
Epoch 245
Validation prc-auc = 0.416382
Epoch 246
Validation prc-auc = 0.417414
Epoch 247
Validation prc-auc = 0.432197
Epoch 248
Validation prc-auc = 0.422332
Epoch 249
Loss = 7.7722e-02, PNorm = 97.2796, GNorm = 2.6706, lr_0 = 6.6056e-04
Validation prc-auc = 0.418751
Epoch 250
Validation prc-auc = 0.366470
Epoch 251
Validation prc-auc = 0.412273
Epoch 252
Validation prc-auc = 0.424434
Epoch 253
Validation prc-auc = 0.427933
Epoch 254
Loss = 2.8538e-02, PNorm = 97.4660, GNorm = 1.2140, lr_0 = 6.5491e-04
Validation prc-auc = 0.429812
Epoch 255
Validation prc-auc = 0.430820
Epoch 256
Validation prc-auc = 0.433818
Epoch 257
Validation prc-auc = 0.430574
Epoch 258
Validation prc-auc = 0.411453
Epoch 259
Loss = 3.5091e-03, PNorm = 97.6146, GNorm = 0.0527, lr_0 = 6.4931e-04
Validation prc-auc = 0.404777
Epoch 260
Validation prc-auc = 0.347160
Epoch 261
Validation prc-auc = 0.306515
Epoch 262
Validation prc-auc = 0.366957
Epoch 263
Validation prc-auc = 0.415866
Epoch 264
Loss = 3.8935e-02, PNorm = 97.7612, GNorm = 0.1609, lr_0 = 6.4376e-04
Validation prc-auc = 0.408930
Epoch 265
Validation prc-auc = 0.410854
Epoch 266
Validation prc-auc = 0.400503
Epoch 267
Validation prc-auc = 0.410132
Epoch 268
Validation prc-auc = 0.412390
Epoch 269
Loss = 4.0784e-02, PNorm = 97.8890, GNorm = 1.4504, lr_0 = 6.3826e-04
Validation prc-auc = 0.416635
Epoch 270
Validation prc-auc = 0.421765
Epoch 271
Validation prc-auc = 0.421664
Epoch 272
Validation prc-auc = 0.424584
Epoch 273
Validation prc-auc = 0.395483
Epoch 274
Loss = 4.5610e-02, PNorm = 98.0335, GNorm = 1.2982, lr_0 = 6.3280e-04
Validation prc-auc = 0.408854
Epoch 275
Validation prc-auc = 0.418384
Epoch 276
Validation prc-auc = 0.424985
Epoch 277
Validation prc-auc = 0.429438
Epoch 278
Validation prc-auc = 0.422713
Epoch 279
Loss = 1.5764e-02, PNorm = 98.1758, GNorm = 0.0342, lr_0 = 6.2739e-04
Validation prc-auc = 0.418030
Epoch 280
Validation prc-auc = 0.405582
Epoch 281
Validation prc-auc = 0.388930
Epoch 282
Validation prc-auc = 0.374751
Epoch 283
Validation prc-auc = 0.385298
Epoch 284
Loss = 1.9619e-02, PNorm = 98.2914, GNorm = 1.4334, lr_0 = 6.2203e-04
Validation prc-auc = 0.386971
Epoch 285
Validation prc-auc = 0.392948
Epoch 286
Validation prc-auc = 0.400615
Epoch 287
Validation prc-auc = 0.413727
Epoch 288
Validation prc-auc = 0.427724
Epoch 289
Loss = 4.5973e-03, PNorm = 98.3934, GNorm = 0.2219, lr_0 = 6.1671e-04
Validation prc-auc = 0.428551
Epoch 290
Validation prc-auc = 0.431270
Epoch 291
Validation prc-auc = 0.430121
Epoch 292
Validation prc-auc = 0.435024
Epoch 293
Validation prc-auc = 0.433014
Epoch 294
Loss = 1.2910e-03, PNorm = 98.4690, GNorm = 0.0697, lr_0 = 6.1144e-04
Validation prc-auc = 0.429572
Epoch 295
Validation prc-auc = 0.429458
Epoch 296
Validation prc-auc = 0.428774
Epoch 297
Validation prc-auc = 0.432749
Epoch 298
Validation prc-auc = 0.432662
Epoch 299
Loss = 3.4191e-03, PNorm = 98.5224, GNorm = 0.3035, lr_0 = 6.0621e-04
Validation prc-auc = 0.432138
Model 0 best validation prc-auc = 0.476246 on epoch 145
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.236615
Ensemble test prc-auc = 0.236615
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.361053
Epoch 1
Validation prc-auc = 0.341266
Epoch 2
Validation prc-auc = 0.369776
Epoch 3
Validation prc-auc = 0.168239
Epoch 4
Loss = 5.9850e-01, PNorm = 87.5360, GNorm = 1.9351, lr_0 = 6.5000e-04
Validation prc-auc = 0.342158
Epoch 5
Validation prc-auc = 0.389251
Epoch 6
Validation prc-auc = 0.369428
Epoch 7
Validation prc-auc = 0.337913
Epoch 8
Validation prc-auc = 0.165633
Epoch 9
Loss = 4.7876e-01, PNorm = 87.8959, GNorm = 2.3704, lr_0 = 9.9743e-04
Validation prc-auc = 0.330103
Epoch 10
Validation prc-auc = 0.410894
Epoch 11
Validation prc-auc = 0.385421
Epoch 12
Validation prc-auc = 0.303852
Epoch 13
Validation prc-auc = 0.361016
Epoch 14
Loss = 5.4695e-01, PNorm = 88.2818, GNorm = 2.7808, lr_0 = 9.8890e-04
Validation prc-auc = 0.377353
Epoch 15
Validation prc-auc = 0.310827
Epoch 16
Validation prc-auc = 0.329184
Epoch 17
Validation prc-auc = 0.374471
Epoch 18
Validation prc-auc = 0.344668
Epoch 19
Loss = 3.9896e-01, PNorm = 88.5721, GNorm = 1.9923, lr_0 = 9.8045e-04
Validation prc-auc = 0.367554
Epoch 20
Validation prc-auc = 0.321184
Epoch 21
Validation prc-auc = 0.419453
Epoch 22
Validation prc-auc = 0.373738
Epoch 23
Validation prc-auc = 0.244683
Epoch 24
Loss = 3.7026e-01, PNorm = 88.8061, GNorm = 2.7035, lr_0 = 9.7207e-04
Validation prc-auc = 0.313796
Epoch 25
Validation prc-auc = 0.367108
Epoch 26
Validation prc-auc = 0.372203
Epoch 27
Validation prc-auc = 0.283320
Epoch 28
Validation prc-auc = 0.338694
Epoch 29
Loss = 3.1270e-01, PNorm = 89.0280, GNorm = 1.6241, lr_0 = 9.6376e-04
Validation prc-auc = 0.384776
Epoch 30
Validation prc-auc = 0.441958
Epoch 31
Validation prc-auc = 0.437462
Epoch 32
Validation prc-auc = 0.411656
Epoch 33
Validation prc-auc = 0.353645
Epoch 34
Loss = 4.4634e-01, PNorm = 89.2517, GNorm = 1.9478, lr_0 = 9.5552e-04
Validation prc-auc = 0.432705
Epoch 35
Validation prc-auc = 0.372747
Epoch 36
Validation prc-auc = 0.338839
Epoch 37
Validation prc-auc = 0.329412
Epoch 38
Validation prc-auc = 0.394716
Epoch 39
Loss = 2.4228e-01, PNorm = 89.4603, GNorm = 1.6077, lr_0 = 9.4735e-04
Validation prc-auc = 0.413926
Epoch 40
Validation prc-auc = 0.435128
Epoch 41
Validation prc-auc = 0.389737
Epoch 42
Validation prc-auc = 0.318694
Epoch 43
Validation prc-auc = 0.353088
Epoch 44
Loss = 3.5343e-01, PNorm = 89.6739, GNorm = 1.2251, lr_0 = 9.3925e-04
Validation prc-auc = 0.453071
Epoch 45
Validation prc-auc = 0.459411
Epoch 46
Validation prc-auc = 0.448044
Epoch 47
Validation prc-auc = 0.438558
Epoch 48
Validation prc-auc = 0.432726
Epoch 49
Loss = 1.3188e-01, PNorm = 89.8886, GNorm = 0.8375, lr_0 = 9.3122e-04
Validation prc-auc = 0.437770
Epoch 50
Validation prc-auc = 0.424516
Epoch 51
Validation prc-auc = 0.414685
Epoch 52
Validation prc-auc = 0.357644
Epoch 53
Validation prc-auc = 0.340725
Epoch 54
Loss = 2.0068e-01, PNorm = 90.1127, GNorm = 2.0276, lr_0 = 9.2326e-04
Validation prc-auc = 0.405438
Epoch 55
Validation prc-auc = 0.457617
Epoch 56
Validation prc-auc = 0.444903
Epoch 57
Validation prc-auc = 0.442190
Epoch 58
Validation prc-auc = 0.426769
Epoch 59
Loss = 1.8327e-01, PNorm = 90.3492, GNorm = 2.9021, lr_0 = 9.1537e-04
Validation prc-auc = 0.397745
Epoch 60
Validation prc-auc = 0.403916
Epoch 61
Validation prc-auc = 0.392164
Epoch 62
Validation prc-auc = 0.432550
Epoch 63
Validation prc-auc = 0.446048
Epoch 64
Loss = 6.8750e-02, PNorm = 90.6235, GNorm = 0.3758, lr_0 = 9.0754e-04
Validation prc-auc = 0.452032
Epoch 65
Validation prc-auc = 0.447512
Epoch 66
Validation prc-auc = 0.428645
Epoch 67
Validation prc-auc = 0.435632
Epoch 68
Validation prc-auc = 0.442695
Epoch 69
Loss = 1.4165e-01, PNorm = 90.8920, GNorm = 2.1618, lr_0 = 8.9978e-04
Validation prc-auc = 0.443395
Epoch 70
Validation prc-auc = 0.449027
Epoch 71
Validation prc-auc = 0.447290
Epoch 72
Validation prc-auc = 0.461802
Epoch 73
Validation prc-auc = 0.463087
Epoch 74
Loss = 3.8706e-02, PNorm = 91.1633, GNorm = 0.4190, lr_0 = 8.9209e-04
Validation prc-auc = 0.446799
Epoch 75
Validation prc-auc = 0.441396
Epoch 76
Validation prc-auc = 0.447987
Epoch 77
Validation prc-auc = 0.432602
Epoch 78
Validation prc-auc = 0.420063
Epoch 79
Loss = 2.7427e-01, PNorm = 91.4209, GNorm = 2.9367, lr_0 = 8.8447e-04
Validation prc-auc = 0.435565
Epoch 80
Validation prc-auc = 0.453254
Epoch 81
Validation prc-auc = 0.458908
Epoch 82
Validation prc-auc = 0.476077
Epoch 83
Validation prc-auc = 0.483342
Epoch 84
Loss = 1.9344e-01, PNorm = 91.6860, GNorm = 1.4234, lr_0 = 8.7691e-04
Validation prc-auc = 0.485755
Epoch 85
Validation prc-auc = 0.453165
Epoch 86
Validation prc-auc = 0.402680
Epoch 87
Validation prc-auc = 0.390110
Epoch 88
Validation prc-auc = 0.406363
Epoch 89
Loss = 1.7339e-01, PNorm = 91.9594, GNorm = 0.3691, lr_0 = 8.6941e-04
Validation prc-auc = 0.434948
Epoch 90
Validation prc-auc = 0.438840
Epoch 91
Validation prc-auc = 0.427361
Epoch 92
Validation prc-auc = 0.441899
Epoch 93
Validation prc-auc = 0.446124
Epoch 94
Loss = 1.0456e-01, PNorm = 92.2163, GNorm = 1.3788, lr_0 = 8.6198e-04
Validation prc-auc = 0.436195
Epoch 95
Validation prc-auc = 0.433468
Epoch 96
Validation prc-auc = 0.441338
Epoch 97
Validation prc-auc = 0.443452
Epoch 98
Validation prc-auc = 0.445021
Epoch 99
Loss = 1.5105e-01, PNorm = 92.4465, GNorm = 1.7049, lr_0 = 8.5461e-04
Validation prc-auc = 0.438909
Epoch 100
Validation prc-auc = 0.387573
Epoch 101
Validation prc-auc = 0.267256
Epoch 102
Validation prc-auc = 0.282602
Epoch 103
Validation prc-auc = 0.407765
Epoch 104
Loss = 6.3244e-02, PNorm = 92.7249, GNorm = 0.8356, lr_0 = 8.4730e-04
Validation prc-auc = 0.413845
Epoch 105
Validation prc-auc = 0.414722
Epoch 106
Validation prc-auc = 0.450176
Epoch 107
Validation prc-auc = 0.450190
Epoch 108
Validation prc-auc = 0.445502
Epoch 109
Loss = 6.1343e-02, PNorm = 92.9956, GNorm = 0.7824, lr_0 = 8.4006e-04
Validation prc-auc = 0.439098
Epoch 110
Validation prc-auc = 0.436075
Epoch 111
Validation prc-auc = 0.452966
Epoch 112
Validation prc-auc = 0.437333
Epoch 113
Validation prc-auc = 0.450080
Epoch 114
Loss = 1.2794e-01, PNorm = 93.2495, GNorm = 1.3521, lr_0 = 8.3288e-04
Validation prc-auc = 0.445787
Epoch 115
Validation prc-auc = 0.441209
Epoch 116
Validation prc-auc = 0.445625
Epoch 117
Validation prc-auc = 0.442557
Epoch 118
Validation prc-auc = 0.451239
Epoch 119
Loss = 7.3263e-02, PNorm = 93.4627, GNorm = 1.1349, lr_0 = 8.2576e-04
Validation prc-auc = 0.456431
Epoch 120
Validation prc-auc = 0.448005
Epoch 121
Validation prc-auc = 0.446945
Epoch 122
Validation prc-auc = 0.444546
Epoch 123
Validation prc-auc = 0.469649
Epoch 124
Loss = 4.4595e-02, PNorm = 93.6589, GNorm = 0.8077, lr_0 = 8.1870e-04
Validation prc-auc = 0.466159
Epoch 125
Validation prc-auc = 0.460512
Epoch 126
Validation prc-auc = 0.465936
Epoch 127
Validation prc-auc = 0.472048
Epoch 128
Validation prc-auc = 0.468971
Epoch 129
Loss = 1.4512e-02, PNorm = 93.8344, GNorm = 0.5858, lr_0 = 8.1170e-04
Validation prc-auc = 0.458712
Epoch 130
Validation prc-auc = 0.473103
Epoch 131
Validation prc-auc = 0.453370
Epoch 132
Validation prc-auc = 0.459433
Epoch 133
Validation prc-auc = 0.436230
Epoch 134
Loss = 1.3963e-02, PNorm = 94.0305, GNorm = 0.7407, lr_0 = 8.0476e-04
Validation prc-auc = 0.414737
Epoch 135
Validation prc-auc = 0.392736
Epoch 136
Validation prc-auc = 0.355822
Epoch 137
Validation prc-auc = 0.410661
Epoch 138
Validation prc-auc = 0.420467
Epoch 139
Loss = 2.0193e-02, PNorm = 94.4016, GNorm = 0.2071, lr_0 = 7.9788e-04
Validation prc-auc = 0.447047
Epoch 140
Validation prc-auc = 0.433282
Epoch 141
Validation prc-auc = 0.454267
Epoch 142
Validation prc-auc = 0.468251
Epoch 143
Validation prc-auc = 0.457528
Epoch 144
Loss = 7.1579e-02, PNorm = 94.7607, GNorm = 0.5164, lr_0 = 7.9106e-04
Validation prc-auc = 0.449275
Epoch 145
Validation prc-auc = 0.419900
Epoch 146
Validation prc-auc = 0.451166
Epoch 147
Validation prc-auc = 0.443904
Epoch 148
Validation prc-auc = 0.459497
Epoch 149
Loss = 4.8515e-02, PNorm = 95.0268, GNorm = 1.8915, lr_0 = 7.8430e-04
Validation prc-auc = 0.466173
Epoch 150
Validation prc-auc = 0.442829
Epoch 151
Validation prc-auc = 0.445425
Epoch 152
Validation prc-auc = 0.449516
Epoch 153
Validation prc-auc = 0.424516
Epoch 154
Loss = 2.0725e-02, PNorm = 95.2344, GNorm = 0.4501, lr_0 = 7.7759e-04
Validation prc-auc = 0.452034
Epoch 155
Validation prc-auc = 0.445153
Epoch 156
Validation prc-auc = 0.455877
Epoch 157
Validation prc-auc = 0.441912
Epoch 158
Validation prc-auc = 0.471873
Epoch 159
Loss = 9.7265e-02, PNorm = 95.4137, GNorm = 1.9869, lr_0 = 7.7095e-04
Validation prc-auc = 0.465342
Epoch 160
Validation prc-auc = 0.451540
Epoch 161
Validation prc-auc = 0.433803
Epoch 162
Validation prc-auc = 0.428707
Epoch 163
Validation prc-auc = 0.423789
Epoch 164
Loss = 1.0338e-01, PNorm = 95.6449, GNorm = 0.9557, lr_0 = 7.6436e-04
Validation prc-auc = 0.422564
Epoch 165
Validation prc-auc = 0.429690
Epoch 166
Validation prc-auc = 0.431676
Epoch 167
Validation prc-auc = 0.437645
Epoch 168
Validation prc-auc = 0.435470
Epoch 169
Loss = 1.3698e-02, PNorm = 95.8916, GNorm = 0.2672, lr_0 = 7.5782e-04
Validation prc-auc = 0.429422
Epoch 170
Validation prc-auc = 0.429282
Epoch 171
Validation prc-auc = 0.413741
Epoch 172
Validation prc-auc = 0.414921
Epoch 173
Validation prc-auc = 0.415681
Epoch 174
Loss = 5.0007e-02, PNorm = 96.0369, GNorm = 1.1467, lr_0 = 7.5134e-04
Validation prc-auc = 0.415425
Epoch 175
Validation prc-auc = 0.423255
Epoch 176
Validation prc-auc = 0.420689
Epoch 177
Validation prc-auc = 0.400948
Epoch 178
Validation prc-auc = 0.420463
Epoch 179
Loss = 3.8406e-02, PNorm = 96.1246, GNorm = 0.2354, lr_0 = 7.4492e-04
Validation prc-auc = 0.445752
Epoch 180
Validation prc-auc = 0.460330
Epoch 181
Validation prc-auc = 0.458004
Epoch 182
Validation prc-auc = 0.448689
Epoch 183
Validation prc-auc = 0.448059
Epoch 184
Loss = 1.2738e-02, PNorm = 96.2003, GNorm = 0.1187, lr_0 = 7.3855e-04
Validation prc-auc = 0.432160
Epoch 185
Validation prc-auc = 0.425597
Epoch 186
Validation prc-auc = 0.416452
Epoch 187
Validation prc-auc = 0.416781
Epoch 188
Validation prc-auc = 0.435294
Epoch 189
Loss = 5.8447e-02, PNorm = 96.2823, GNorm = 0.2950, lr_0 = 7.3224e-04
Validation prc-auc = 0.446864
Epoch 190
Validation prc-auc = 0.448995
Epoch 191
Validation prc-auc = 0.456215
Epoch 192
Validation prc-auc = 0.462125
Epoch 193
Validation prc-auc = 0.465118
Epoch 194
Loss = 4.6558e-03, PNorm = 96.3777, GNorm = 0.1145, lr_0 = 7.2598e-04
Validation prc-auc = 0.465055
Epoch 195
Validation prc-auc = 0.470555
Epoch 196
Validation prc-auc = 0.469522
Epoch 197
Validation prc-auc = 0.474430
Epoch 198
Validation prc-auc = 0.468164
Epoch 199
Loss = 1.5440e-02, PNorm = 96.4595, GNorm = 0.5249, lr_0 = 7.1977e-04
Validation prc-auc = 0.465640
Epoch 200
Validation prc-auc = 0.466035
Epoch 201
Validation prc-auc = 0.465653
Epoch 202
Validation prc-auc = 0.463467
Epoch 203
Validation prc-auc = 0.449225
Epoch 204
Loss = 1.5921e-02, PNorm = 96.5361, GNorm = 0.0925, lr_0 = 7.1362e-04
Validation prc-auc = 0.446083
Epoch 205
Validation prc-auc = 0.438305
Epoch 206
Validation prc-auc = 0.442953
Epoch 207
Validation prc-auc = 0.450216
Epoch 208
Validation prc-auc = 0.448946
Epoch 209
Loss = 6.6780e-03, PNorm = 96.6102, GNorm = 0.2742, lr_0 = 7.0752e-04
Validation prc-auc = 0.449399
Epoch 210
Validation prc-auc = 0.447763
Epoch 211
Validation prc-auc = 0.452280
Epoch 212
Validation prc-auc = 0.455401
Epoch 213
Validation prc-auc = 0.452431
Epoch 214
Loss = 2.6141e-03, PNorm = 96.6768, GNorm = 0.1459, lr_0 = 7.0147e-04
Validation prc-auc = 0.458967
Epoch 215
Validation prc-auc = 0.449467
Epoch 216
Validation prc-auc = 0.449012
Epoch 217
Validation prc-auc = 0.447392
Epoch 218
Validation prc-auc = 0.446829
Epoch 219
Loss = 4.5114e-02, PNorm = 96.7362, GNorm = 0.0231, lr_0 = 6.9548e-04
Validation prc-auc = 0.445869
Epoch 220
Validation prc-auc = 0.443775
Epoch 221
Validation prc-auc = 0.444655
Epoch 222
Validation prc-auc = 0.441334
Epoch 223
Validation prc-auc = 0.429779
Epoch 224
Loss = 5.5443e-03, PNorm = 96.8311, GNorm = 0.2639, lr_0 = 6.8953e-04
Validation prc-auc = 0.423030
Epoch 225
Validation prc-auc = 0.421738
Epoch 226
Validation prc-auc = 0.419184
Epoch 227
Validation prc-auc = 0.418907
Epoch 228
Validation prc-auc = 0.432340
Epoch 229
Loss = 1.5016e-03, PNorm = 96.9190, GNorm = 0.0232, lr_0 = 6.8364e-04
Validation prc-auc = 0.437396
Epoch 230
Validation prc-auc = 0.441680
Epoch 231
Validation prc-auc = 0.444071
Epoch 232
Validation prc-auc = 0.438644
Epoch 233
Validation prc-auc = 0.440065
Epoch 234
Loss = 2.4113e-02, PNorm = 96.9916, GNorm = 0.9091, lr_0 = 6.7779e-04
Validation prc-auc = 0.440850
Epoch 235
Validation prc-auc = 0.444379
Epoch 236
Validation prc-auc = 0.434562
Epoch 237
Validation prc-auc = 0.440440
Epoch 238
Validation prc-auc = 0.442800
Epoch 239
Loss = 2.7161e-02, PNorm = 97.0689, GNorm = 1.5102, lr_0 = 6.7200e-04
Validation prc-auc = 0.444278
Epoch 240
Validation prc-auc = 0.446505
Epoch 241
Validation prc-auc = 0.446955
Epoch 242
Validation prc-auc = 0.459915
Epoch 243
Validation prc-auc = 0.446516
Epoch 244
Loss = 9.4209e-04, PNorm = 97.1345, GNorm = 0.0279, lr_0 = 6.6625e-04
Validation prc-auc = 0.440345
Epoch 245
Validation prc-auc = 0.439792
Epoch 246
Validation prc-auc = 0.441016
Epoch 247
Validation prc-auc = 0.440846
Epoch 248
Validation prc-auc = 0.440089
Epoch 249
Loss = 5.7397e-03, PNorm = 97.2113, GNorm = 0.3553, lr_0 = 6.6056e-04
Validation prc-auc = 0.447528
Epoch 250
Validation prc-auc = 0.449710
Epoch 251
Validation prc-auc = 0.449122
Epoch 252
Validation prc-auc = 0.438457
Epoch 253
Validation prc-auc = 0.434559
Epoch 254
Loss = 1.8907e-03, PNorm = 97.2879, GNorm = 0.0310, lr_0 = 6.5491e-04
Validation prc-auc = 0.431736
Epoch 255
Validation prc-auc = 0.431194
Epoch 256
Validation prc-auc = 0.431104
Epoch 257
Validation prc-auc = 0.430434
Epoch 258
Validation prc-auc = 0.429894
Epoch 259
Loss = 5.6587e-04, PNorm = 97.3418, GNorm = 0.0222, lr_0 = 6.4931e-04
Validation prc-auc = 0.430449
Epoch 260
Validation prc-auc = 0.431480
Epoch 261
Validation prc-auc = 0.448319
Epoch 262
Validation prc-auc = 0.448852
Epoch 263
Validation prc-auc = 0.449328
Epoch 264
Loss = 2.6736e-02, PNorm = 97.3918, GNorm = 0.9386, lr_0 = 6.4376e-04
Validation prc-auc = 0.448704
Epoch 265
Validation prc-auc = 0.446994
Epoch 266
Validation prc-auc = 0.440040
Epoch 267
Validation prc-auc = 0.432106
Epoch 268
Validation prc-auc = 0.431363
Epoch 269
Loss = 4.5447e-03, PNorm = 97.4475, GNorm = 0.1051, lr_0 = 6.3826e-04
Validation prc-auc = 0.417102
Epoch 270
Validation prc-auc = 0.416652
Epoch 271
Validation prc-auc = 0.414419
Epoch 272
Validation prc-auc = 0.413494
Epoch 273
Validation prc-auc = 0.427818
Epoch 274
Loss = 1.6593e-02, PNorm = 97.4884, GNorm = 1.0494, lr_0 = 6.3280e-04
Validation prc-auc = 0.435438
Epoch 275
Validation prc-auc = 0.420458
Epoch 276
Validation prc-auc = 0.434786
Epoch 277
Validation prc-auc = 0.430399
Epoch 278
Validation prc-auc = 0.430766
Epoch 279
Loss = 1.2104e-02, PNorm = 97.5404, GNorm = 0.6629, lr_0 = 6.2739e-04
Validation prc-auc = 0.433093
Epoch 280
Validation prc-auc = 0.433681
Epoch 281
Validation prc-auc = 0.439658
Epoch 282
Validation prc-auc = 0.437709
Epoch 283
Validation prc-auc = 0.437724
Epoch 284
Loss = 4.8312e-03, PNorm = 97.6181, GNorm = 0.4381, lr_0 = 6.2203e-04
Validation prc-auc = 0.431840
Epoch 285
Validation prc-auc = 0.432292
Epoch 286
Validation prc-auc = 0.436913
Epoch 287
Validation prc-auc = 0.443821
Epoch 288
Validation prc-auc = 0.443180
Epoch 289
Loss = 7.0164e-04, PNorm = 97.6983, GNorm = 0.0351, lr_0 = 6.1671e-04
Validation prc-auc = 0.437048
Epoch 290
Validation prc-auc = 0.438999
Epoch 291
Validation prc-auc = 0.444752
Epoch 292
Validation prc-auc = 0.451981
Epoch 293
Validation prc-auc = 0.451331
Epoch 294
Loss = 1.3788e-01, PNorm = 97.7837, GNorm = 0.1147, lr_0 = 6.1144e-04
Validation prc-auc = 0.457891
Epoch 295
Validation prc-auc = 0.454775
Epoch 296
Validation prc-auc = 0.449256
Epoch 297
Validation prc-auc = 0.440693
Epoch 298
Validation prc-auc = 0.434153
Epoch 299
Loss = 2.0858e-03, PNorm = 97.8828, GNorm = 0.0539, lr_0 = 6.0621e-04
Validation prc-auc = 0.430308
Model 0 best validation prc-auc = 0.485755 on epoch 84
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.225148
Ensemble test prc-auc = 0.225148
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.25, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.25, inplace=False)
    (1): Linear(in_features=2028, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.25, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 13,881,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.417140
Epoch 1
Validation prc-auc = 0.361958
Epoch 2
Validation prc-auc = 0.387800
Epoch 3
Validation prc-auc = 0.301190
Epoch 4
Loss = 5.7424e-01, PNorm = 87.5281, GNorm = 1.6243, lr_0 = 6.5000e-04
Validation prc-auc = 0.309369
Epoch 5
Validation prc-auc = 0.401356
Epoch 6
Validation prc-auc = 0.424202
Epoch 7
Validation prc-auc = 0.367308
Epoch 8
Validation prc-auc = 0.333376
Epoch 9
Loss = 4.3440e-01, PNorm = 87.8609, GNorm = 1.1686, lr_0 = 9.9743e-04
Validation prc-auc = 0.411369
Epoch 10
Validation prc-auc = 0.405103
Epoch 11
Validation prc-auc = 0.341350
Epoch 12
Validation prc-auc = 0.223800
Epoch 13
Validation prc-auc = 0.220977
Epoch 14
Loss = 3.2674e-01, PNorm = 88.2251, GNorm = 1.3707, lr_0 = 9.8890e-04
Validation prc-auc = 0.302613
Epoch 15
Validation prc-auc = 0.444337
Epoch 16
Validation prc-auc = 0.396699
Epoch 17
Validation prc-auc = 0.334616
Epoch 18
Validation prc-auc = 0.316324
Epoch 19
Loss = 3.0722e-01, PNorm = 88.4961, GNorm = 1.0472, lr_0 = 9.8045e-04
Validation prc-auc = 0.330910
Epoch 20
Validation prc-auc = 0.324143
Epoch 21
Validation prc-auc = 0.191866
Epoch 22
Validation prc-auc = 0.180547
Epoch 23
Validation prc-auc = 0.377225
Epoch 24
Loss = 3.2914e-01, PNorm = 88.7276, GNorm = 2.6338, lr_0 = 9.7207e-04
Validation prc-auc = 0.438449
Epoch 25
Validation prc-auc = 0.408651
Epoch 26
Validation prc-auc = 0.257485
Epoch 27
Validation prc-auc = 0.234576
Epoch 28
Validation prc-auc = 0.284924
Epoch 29
Loss = 2.0311e-01, PNorm = 88.9412, GNorm = 1.6457, lr_0 = 9.6376e-04
Validation prc-auc = 0.388726
Epoch 30
Validation prc-auc = 0.407979
Epoch 31
Validation prc-auc = 0.401033
Epoch 32
Validation prc-auc = 0.308665
Epoch 33
Validation prc-auc = 0.296208
Epoch 34
Loss = 2.6446e-01, PNorm = 89.1324, GNorm = 1.8819, lr_0 = 9.5552e-04
Validation prc-auc = 0.377700
Epoch 35
Validation prc-auc = 0.410499
Epoch 36
Validation prc-auc = 0.421610
Epoch 37
Validation prc-auc = 0.409955
Epoch 38
Validation prc-auc = 0.255347
Epoch 39
Loss = 3.9790e-01, PNorm = 89.3400, GNorm = 1.7193, lr_0 = 9.4735e-04
Validation prc-auc = 0.346894
Epoch 40
Validation prc-auc = 0.411953
Epoch 41
Validation prc-auc = 0.414318
Epoch 42
Validation prc-auc = 0.335614
Epoch 43
Validation prc-auc = 0.408741
Epoch 44
Loss = 3.3759e-01, PNorm = 89.5522, GNorm = 1.6676, lr_0 = 9.3925e-04
Validation prc-auc = 0.441099
Epoch 45
Validation prc-auc = 0.422537
Epoch 46
Validation prc-auc = 0.406808
Epoch 47
Validation prc-auc = 0.424765
Epoch 48
Validation prc-auc = 0.437940
Epoch 49
Loss = 1.7707e-01, PNorm = 89.7788, GNorm = 1.0476, lr_0 = 9.3122e-04
Validation prc-auc = 0.398623
Epoch 50
Validation prc-auc = 0.368975
Epoch 51
Validation prc-auc = 0.338789
Epoch 52
Validation prc-auc = 0.377931
Epoch 53
Validation prc-auc = 0.406067
Epoch 54
Loss = 1.6108e-01, PNorm = 89.9811, GNorm = 1.1832, lr_0 = 9.2326e-04
Validation prc-auc = 0.448953
Epoch 55
Validation prc-auc = 0.449051
Epoch 56
Validation prc-auc = 0.452097
Epoch 57
Validation prc-auc = 0.417428
Epoch 58
Validation prc-auc = 0.383613
Epoch 59
Loss = 1.3460e-01, PNorm = 90.1801, GNorm = 1.1715, lr_0 = 9.1537e-04
Validation prc-auc = 0.412585
Epoch 60
Validation prc-auc = 0.439510
Epoch 61
Validation prc-auc = 0.462513
Epoch 62
Validation prc-auc = 0.457531
Epoch 63
Validation prc-auc = 0.436498
Epoch 64
Loss = 2.5492e-01, PNorm = 90.4064, GNorm = 1.5336, lr_0 = 9.0754e-04
Validation prc-auc = 0.247897
Epoch 65
Validation prc-auc = 0.228948
Epoch 66
Validation prc-auc = 0.413245
Epoch 67
Validation prc-auc = 0.438112
Epoch 68
Validation prc-auc = 0.438524
Epoch 69
Loss = 1.5333e-01, PNorm = 90.6801, GNorm = 1.0088, lr_0 = 8.9978e-04
Validation prc-auc = 0.437867
Epoch 70
Validation prc-auc = 0.434937
Epoch 71
Validation prc-auc = 0.457155
Epoch 72
Validation prc-auc = 0.453162
Epoch 73
Validation prc-auc = 0.438815
Epoch 74
Loss = 1.6745e-01, PNorm = 91.0130, GNorm = 2.3185, lr_0 = 8.9209e-04
Validation prc-auc = 0.439111
Epoch 75
Validation prc-auc = 0.469888
Epoch 76
Validation prc-auc = 0.428702
Epoch 77
Validation prc-auc = 0.406168
Epoch 78
Validation prc-auc = 0.363124
Epoch 79
Loss = 2.0075e-01, PNorm = 91.3228, GNorm = 1.9966, lr_0 = 8.8447e-04
Validation prc-auc = 0.339289
Epoch 80
Validation prc-auc = 0.377811
Epoch 81
Validation prc-auc = 0.423811
Epoch 82
Validation prc-auc = 0.440166
Epoch 83
Validation prc-auc = 0.442499
Epoch 84
Loss = 1.1807e-01, PNorm = 91.6440, GNorm = 0.2353, lr_0 = 8.7691e-04
Validation prc-auc = 0.423829
Epoch 85
Validation prc-auc = 0.399310
Epoch 86
Validation prc-auc = 0.358954
Epoch 87
Validation prc-auc = 0.354093
Epoch 88
Validation prc-auc = 0.379405
Epoch 89
Loss = 6.3441e-02, PNorm = 91.9001, GNorm = 0.5722, lr_0 = 8.6941e-04
Validation prc-auc = 0.391966
Epoch 90
Validation prc-auc = 0.394247
Epoch 91
Validation prc-auc = 0.340885
Epoch 92
Validation prc-auc = 0.343384
Epoch 93
Validation prc-auc = 0.399874
Epoch 94
Loss = 3.9739e-02, PNorm = 92.1315, GNorm = 0.5940, lr_0 = 8.6198e-04
Validation prc-auc = 0.390638
Epoch 95
Validation prc-auc = 0.392678
Epoch 96
Validation prc-auc = 0.393138
Epoch 97
Validation prc-auc = 0.379227
Epoch 98
Validation prc-auc = 0.369496
Epoch 99
Loss = 8.5410e-02, PNorm = 92.3733, GNorm = 0.7788, lr_0 = 8.5461e-04
Validation prc-auc = 0.324130
Epoch 100
Validation prc-auc = 0.325716
Epoch 101
Validation prc-auc = 0.369108
Epoch 102
Validation prc-auc = 0.432118
Epoch 103
Validation prc-auc = 0.438524
Epoch 104
Loss = 2.4597e-02, PNorm = 92.6301, GNorm = 0.3526, lr_0 = 8.4730e-04
Validation prc-auc = 0.431415
Epoch 105
Validation prc-auc = 0.418388
Epoch 106
Validation prc-auc = 0.408982
Epoch 107
Validation prc-auc = 0.420260
Epoch 108
Validation prc-auc = 0.399368
Epoch 109
Loss = 1.0514e-01, PNorm = 92.9941, GNorm = 2.0803, lr_0 = 8.4006e-04
Validation prc-auc = 0.429934
Epoch 110
Validation prc-auc = 0.433791
Epoch 111
Validation prc-auc = 0.429609
Epoch 112
Validation prc-auc = 0.427905
Epoch 113
Validation prc-auc = 0.427725
Epoch 114
Loss = 1.3224e-01, PNorm = 93.3575, GNorm = 0.8978, lr_0 = 8.3288e-04
Validation prc-auc = 0.424659
Epoch 115
Validation prc-auc = 0.410582
Epoch 116
Validation prc-auc = 0.408663
Epoch 117
Validation prc-auc = 0.424438
Epoch 118
Validation prc-auc = 0.424246
Epoch 119
Loss = 1.3215e-02, PNorm = 93.6539, GNorm = 0.3135, lr_0 = 8.2576e-04
Validation prc-auc = 0.427202
Epoch 120
Validation prc-auc = 0.430112
Epoch 121
Validation prc-auc = 0.424454
Epoch 122
Validation prc-auc = 0.425858
Epoch 123
Validation prc-auc = 0.435534
Epoch 124
Loss = 3.1534e-02, PNorm = 93.8841, GNorm = 0.7496, lr_0 = 8.1870e-04
Validation prc-auc = 0.424479
Epoch 125
Validation prc-auc = 0.436256
Epoch 126
Validation prc-auc = 0.422203
Epoch 127
Validation prc-auc = 0.419243
Epoch 128
Validation prc-auc = 0.429155
Epoch 129
Loss = 2.1779e-02, PNorm = 94.0482, GNorm = 0.5217, lr_0 = 8.1170e-04
Validation prc-auc = 0.424719
Epoch 130
Validation prc-auc = 0.425386
Epoch 131
Validation prc-auc = 0.427926
Epoch 132
Validation prc-auc = 0.403846
Epoch 133
Validation prc-auc = 0.406247
Epoch 134
Loss = 5.4690e-02, PNorm = 94.1811, GNorm = 1.3410, lr_0 = 8.0476e-04
Validation prc-auc = 0.406385
Epoch 135
Validation prc-auc = 0.403933
Epoch 136
Validation prc-auc = 0.387613
Epoch 137
Validation prc-auc = 0.387079
Epoch 138
Validation prc-auc = 0.396793
Epoch 139
Loss = 7.1997e-02, PNorm = 94.2993, GNorm = 1.2889, lr_0 = 7.9788e-04
Validation prc-auc = 0.397805
Epoch 140
Validation prc-auc = 0.408275
Epoch 141
Validation prc-auc = 0.406970
Epoch 142
Validation prc-auc = 0.403552
Epoch 143
Validation prc-auc = 0.406783
Epoch 144
Loss = 2.2263e-02, PNorm = 94.3993, GNorm = 0.6613, lr_0 = 7.9106e-04
Validation prc-auc = 0.420497
Epoch 145
Validation prc-auc = 0.413654
Epoch 146
Validation prc-auc = 0.409042
Epoch 147
Validation prc-auc = 0.400767
Epoch 148
Validation prc-auc = 0.345054
Epoch 149
Loss = 3.4934e-02, PNorm = 94.5664, GNorm = 0.6512, lr_0 = 7.8430e-04
Validation prc-auc = 0.345582
Epoch 150
Validation prc-auc = 0.421546
Epoch 151
Validation prc-auc = 0.428689
Epoch 152
Validation prc-auc = 0.431821
Epoch 153
Validation prc-auc = 0.444955
Epoch 154
Loss = 6.0420e-03, PNorm = 94.7426, GNorm = 0.1992, lr_0 = 7.7759e-04
Validation prc-auc = 0.435413
Epoch 155
Validation prc-auc = 0.413061
Epoch 156
Validation prc-auc = 0.409526
Epoch 157
Validation prc-auc = 0.398699
Epoch 158
Validation prc-auc = 0.394419
Epoch 159
Loss = 1.5177e-02, PNorm = 94.8970, GNorm = 0.7766, lr_0 = 7.7095e-04
Validation prc-auc = 0.390739
Epoch 160
Validation prc-auc = 0.374101
Epoch 161
Validation prc-auc = 0.373868
Epoch 162
Validation prc-auc = 0.397002
Epoch 163
Validation prc-auc = 0.390461
Epoch 164
Loss = 1.6528e-02, PNorm = 95.0792, GNorm = 0.2151, lr_0 = 7.6436e-04
Validation prc-auc = 0.393996
Epoch 165
Validation prc-auc = 0.410628
Epoch 166
Validation prc-auc = 0.426748
Epoch 167
Validation prc-auc = 0.446748
Epoch 168
Validation prc-auc = 0.452823
Epoch 169
Loss = 1.3508e-02, PNorm = 95.2660, GNorm = 0.1860, lr_0 = 7.5782e-04
Validation prc-auc = 0.459417
Epoch 170
Validation prc-auc = 0.447300
Epoch 171
Validation prc-auc = 0.445379
Epoch 172
Validation prc-auc = 0.435257
Epoch 173
Validation prc-auc = 0.440217
Epoch 174
Loss = 3.8871e-03, PNorm = 95.4476, GNorm = 0.0879, lr_0 = 7.5134e-04
Validation prc-auc = 0.453124
Epoch 175
Validation prc-auc = 0.437912
Epoch 176
Validation prc-auc = 0.429761
Epoch 177
Validation prc-auc = 0.438115
Epoch 178
Validation prc-auc = 0.438281
Epoch 179
Loss = 5.6369e-02, PNorm = 95.5730, GNorm = 1.3070, lr_0 = 7.4492e-04
Validation prc-auc = 0.435151
Epoch 180
Validation prc-auc = 0.434225
Epoch 181
Validation prc-auc = 0.433029
Epoch 182
Validation prc-auc = 0.431921
Epoch 183
Validation prc-auc = 0.431736
Epoch 184
Loss = 4.4258e-02, PNorm = 95.7120, GNorm = 1.1465, lr_0 = 7.3855e-04
Validation prc-auc = 0.433572
Epoch 185
Validation prc-auc = 0.420440
Epoch 186
Validation prc-auc = 0.410308
Epoch 187
Validation prc-auc = 0.414740
Epoch 188
Validation prc-auc = 0.421090
Epoch 189
Loss = 5.5449e-03, PNorm = 95.8599, GNorm = 0.0590, lr_0 = 7.3224e-04
Validation prc-auc = 0.440575
Epoch 190
Validation prc-auc = 0.431294
Epoch 191
Validation prc-auc = 0.431398
Epoch 192
Validation prc-auc = 0.423063
Epoch 193
Validation prc-auc = 0.415629
Epoch 194
Loss = 1.1812e-02, PNorm = 95.9736, GNorm = 0.2860, lr_0 = 7.2598e-04
Validation prc-auc = 0.414413
Epoch 195
Validation prc-auc = 0.416390
Epoch 196
Validation prc-auc = 0.418400
Epoch 197
Validation prc-auc = 0.419493
Epoch 198
Validation prc-auc = 0.420379
Epoch 199
Loss = 5.6288e-03, PNorm = 96.0976, GNorm = 0.4576, lr_0 = 7.1977e-04
Validation prc-auc = 0.418279
Epoch 200
Validation prc-auc = 0.410133
Epoch 201
Validation prc-auc = 0.420664
Epoch 202
Validation prc-auc = 0.416214
Epoch 203
Validation prc-auc = 0.402884
Epoch 204
Loss = 6.5907e-03, PNorm = 96.1882, GNorm = 0.0348, lr_0 = 7.1362e-04
Validation prc-auc = 0.408384
Epoch 205
Validation prc-auc = 0.415372
Epoch 206
Validation prc-auc = 0.418133
Epoch 207
Validation prc-auc = 0.426553
Epoch 208
Validation prc-auc = 0.432261
Epoch 209
Loss = 4.0418e-03, PNorm = 96.2838, GNorm = 0.0256, lr_0 = 7.0752e-04
Validation prc-auc = 0.418963
Epoch 210
Validation prc-auc = 0.418786
Epoch 211
Validation prc-auc = 0.417517
Epoch 212
Validation prc-auc = 0.402709
Epoch 213
Validation prc-auc = 0.402705
Epoch 214
Loss = 1.7441e-03, PNorm = 96.3629, GNorm = 0.0814, lr_0 = 7.0147e-04
Validation prc-auc = 0.402872
Epoch 215
Validation prc-auc = 0.393508
Epoch 216
Validation prc-auc = 0.402153
Epoch 217
Validation prc-auc = 0.402798
Epoch 218
Validation prc-auc = 0.413514
Epoch 219
Loss = 5.4792e-03, PNorm = 96.4127, GNorm = 0.4981, lr_0 = 6.9548e-04
Validation prc-auc = 0.414983
Epoch 220
Validation prc-auc = 0.416166
Epoch 221
Validation prc-auc = 0.415095
Epoch 222
Validation prc-auc = 0.415254
Epoch 223
Validation prc-auc = 0.414763
Epoch 224
Loss = 3.7689e-03, PNorm = 96.4496, GNorm = 0.0554, lr_0 = 6.8953e-04
Validation prc-auc = 0.415719
Epoch 225
Validation prc-auc = 0.415634
Epoch 226
Validation prc-auc = 0.421767
Epoch 227
Validation prc-auc = 0.421652
Epoch 228
Validation prc-auc = 0.429290
Epoch 229
Loss = 1.4813e-02, PNorm = 96.4793, GNorm = 0.4054, lr_0 = 6.8364e-04
Validation prc-auc = 0.422222
Epoch 230
Validation prc-auc = 0.417427
Epoch 231
Validation prc-auc = 0.420920
Epoch 232
Validation prc-auc = 0.414452
Epoch 233
Validation prc-auc = 0.411642
Epoch 234
Loss = 6.4631e-04, PNorm = 96.5471, GNorm = 0.0103, lr_0 = 6.7779e-04
Validation prc-auc = 0.414163
Epoch 235
Validation prc-auc = 0.413354
Epoch 236
Validation prc-auc = 0.418500
Epoch 237
Validation prc-auc = 0.418034
Epoch 238
Validation prc-auc = 0.418816
Epoch 239
Loss = 1.5833e-03, PNorm = 96.6479, GNorm = 0.0432, lr_0 = 6.7200e-04
Validation prc-auc = 0.420640
Epoch 240
Validation prc-auc = 0.420861
Epoch 241
Validation prc-auc = 0.422372
Epoch 242
Validation prc-auc = 0.421816
Epoch 243
Validation prc-auc = 0.433373
Epoch 244
Loss = 1.2072e-03, PNorm = 96.7335, GNorm = 0.0797, lr_0 = 6.6625e-04
Validation prc-auc = 0.424415
Epoch 245
Validation prc-auc = 0.417102
Epoch 246
Validation prc-auc = 0.416683
Epoch 247
Validation prc-auc = 0.427204
Epoch 248
Validation prc-auc = 0.445591
Epoch 249
Loss = 3.5025e-02, PNorm = 96.8356, GNorm = 2.3921, lr_0 = 6.6056e-04
Validation prc-auc = 0.449419
Epoch 250
Validation prc-auc = 0.447372
Epoch 251
Validation prc-auc = 0.437450
Epoch 252
Validation prc-auc = 0.450809
Epoch 253
Validation prc-auc = 0.454246
Epoch 254
Loss = 4.2109e-02, PNorm = 96.9557, GNorm = 0.0357, lr_0 = 6.5491e-04
Validation prc-auc = 0.448630
Epoch 255
Validation prc-auc = 0.443481
Epoch 256
Validation prc-auc = 0.449743
Epoch 257
Validation prc-auc = 0.433395
Epoch 258
Validation prc-auc = 0.413667
Epoch 259
Loss = 2.6444e-02, PNorm = 97.1236, GNorm = 1.1146, lr_0 = 6.4931e-04
Validation prc-auc = 0.402947
Epoch 260
Validation prc-auc = 0.403509
Epoch 261
Validation prc-auc = 0.405762
Epoch 262
Validation prc-auc = 0.395690
Epoch 263
Validation prc-auc = 0.396518
Epoch 264
Loss = 2.5626e-03, PNorm = 97.3198, GNorm = 0.0982, lr_0 = 6.4376e-04
Validation prc-auc = 0.395369
Epoch 265
Validation prc-auc = 0.389500
Epoch 266
Validation prc-auc = 0.380033
Epoch 267
Validation prc-auc = 0.396264
Epoch 268
Validation prc-auc = 0.374500
Epoch 269
Loss = 8.7241e-02, PNorm = 97.4662, GNorm = 0.3106, lr_0 = 6.3826e-04
Validation prc-auc = 0.411607
Epoch 270
Validation prc-auc = 0.415287
Epoch 271
Validation prc-auc = 0.421288
Epoch 272
Validation prc-auc = 0.405699
Epoch 273
Validation prc-auc = 0.397295
Epoch 274
Loss = 8.8007e-03, PNorm = 97.6470, GNorm = 0.0526, lr_0 = 6.3280e-04
Validation prc-auc = 0.383915
Epoch 275
Validation prc-auc = 0.373382
Epoch 276
Validation prc-auc = 0.349013
Epoch 277
Validation prc-auc = 0.381906
Epoch 278
Validation prc-auc = 0.396843
Epoch 279
Loss = 8.6782e-02, PNorm = 97.9552, GNorm = 1.8076, lr_0 = 6.2739e-04
Validation prc-auc = 0.406041
Epoch 280
Validation prc-auc = 0.419401
Epoch 281
Validation prc-auc = 0.425145
Epoch 282
Validation prc-auc = 0.425898
Epoch 283
Validation prc-auc = 0.434923
Epoch 284
Loss = 5.0009e-03, PNorm = 98.2525, GNorm = 0.3306, lr_0 = 6.2203e-04
Validation prc-auc = 0.438537
Epoch 285
Validation prc-auc = 0.448739
Epoch 286
Validation prc-auc = 0.452698
Epoch 287
Validation prc-auc = 0.445587
Epoch 288
Validation prc-auc = 0.434430
Epoch 289
Loss = 1.3271e-02, PNorm = 98.4394, GNorm = 0.8546, lr_0 = 6.1671e-04
Validation prc-auc = 0.419658
Epoch 290
Validation prc-auc = 0.404689
Epoch 291
Validation prc-auc = 0.402058
Epoch 292
Validation prc-auc = 0.384752
Epoch 293
Validation prc-auc = 0.419032
Epoch 294
Loss = 1.5030e-02, PNorm = 98.5898, GNorm = 0.4487, lr_0 = 6.1144e-04
Validation prc-auc = 0.420843
Epoch 295
Validation prc-auc = 0.421920
Epoch 296
Validation prc-auc = 0.421879
Epoch 297
Validation prc-auc = 0.416585
Epoch 298
Validation prc-auc = 0.414963
Epoch 299
Loss = 2.5491e-03, PNorm = 98.7005, GNorm = 0.2410, lr_0 = 6.0621e-04
Validation prc-auc = 0.414229
Model 0 best validation prc-auc = 0.469888 on epoch 75
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.339258
Ensemble test prc-auc = 0.339258
10-fold cross validation
	Seed 0 ==> test prc-auc = 0.371892
	Seed 1 ==> test prc-auc = 0.234045
	Seed 2 ==> test prc-auc = 0.213505
	Seed 3 ==> test prc-auc = 0.318417
	Seed 4 ==> test prc-auc = 0.362098
	Seed 5 ==> test prc-auc = 0.200059
	Seed 6 ==> test prc-auc = 0.261625
	Seed 7 ==> test prc-auc = 0.236615
	Seed 8 ==> test prc-auc = 0.225148
	Seed 9 ==> test prc-auc = 0.339258
Overall test prc-auc = 0.276266 +/- 0.061794
Elapsed time = 0:07:06
